>>> kops: Building testing/kops 1.24.0-r0 (using abuild 3.9.0-r2) started Sun, 24 Jul 2022 17:13:29 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.24.3-r1) (2/3) Installing go (1.18.4-r2) (3/3) Installing .makedepends-kops (20220724.171330) Executing busybox-1.35.0-r20.trigger OK: 713 MiB in 104 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.24.0.tar.gz::https://github.com/kubernetes/kops/archive/v1.24.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2384k 0 2384k 0 0 3561k 0 --:--:-- --:--:-- --:--:-- 3561k 100 11.1M 0 11.1M 0 0 6810k 0 --:--:-- 0:00:01 --:--:-- 8981k 100 15.6M 0 15.6M 0 0 6011k 0 --:--:-- 0:00:02 --:--:-- 6833k 100 20.8M 0 20.8M 0 0 5828k 0 --:--:-- 0:00:03 --:--:-- 6336k 100 24.9M 0 24.9M 0 0 5471k 0 --:--:-- 0:00:04 --:--:-- 5791k 100 29.3M 0 29.3M 0 0 5315k 0 --:--:-- 0:00:05 --:--:-- 5551k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.24.0.tar.gz >>> kops: Checking sha512sums... kops-1.24.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.24.0.tar.gz... go: downloading k8s.io/klog/v2 v2.70.0 go: downloading google.golang.org/api v0.86.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v0.25.0 go: downloading github.com/aws/aws-sdk-go v1.44.45 go: downloading k8s.io/apimachinery v0.24.0 go: downloading github.com/cert-manager/cert-manager v1.8.2 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/prometheus/client_golang v1.12.2 go: downloading github.com/spf13/cobra v1.5.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/client-go v0.24.0 go: downloading github.com/spf13/viper v1.12.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/component-base v0.24.0 go: downloading k8s.io/api v0.24.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/digitalocean/godo v1.81.0 go: downloading k8s.io/cli-runtime v0.24.0 go: downloading golang.org/x/oauth2 v0.0.0-20220628200809-02e64fa58f26 go: downloading github.com/go-logr/logr v1.2.3 go: downloading cloud.google.com/go/compute v1.7.0 go: downloading golang.org/x/sys v0.0.0-20220627191245-f75cf1eec38b go: downloading sigs.k8s.io/controller-runtime v0.12.2 go: downloading github.com/google/go-cmp v0.5.8 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.24.0 go: downloading github.com/google/go-containerregistry v0.10.0 go: downloading github.com/stretchr/testify v1.8.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e go: downloading github.com/hetznercloud/hcloud-go v1.35.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.24.0 go: downloading k8s.io/mount-utils v0.24.0 go: downloading k8s.io/utils v0.0.0-20220210201930-3a6ce19ff2f9 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.3.1 go: downloading golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d go: downloading helm.sh/helm/v3 v3.9.0 go: downloading k8s.io/kubectl v0.24.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/fsnotify/fsnotify v1.5.4 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.8.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.32.1 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.1 go: downloading golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.5.7-v3refs go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.66.6 go: downloading github.com/hashicorp/vault/api v1.7.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/Azure/azure-sdk-for-go v66.0.0+incompatible go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.121.0 go: downloading golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.27 go: downloading github.com/google/go-tpm-tools v0.3.8 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.13.0 go: downloading github.com/zclconf/go-cty v1.10.0 go: downloading k8s.io/gengo v0.0.0-20220613173612-397b4ae3bce7 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.24.0 go: downloading github.com/docker/cli v20.10.16+incompatible go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.11.4 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/subosito/gotenv v1.3.0 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.66.6 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.7.3 go: downloading golang.org/x/text v0.3.7 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.11.4 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.6 go: downloading k8s.io/kube-openapi v0.0.0-20220328201542-3ee0da9b0b42 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/json-iterator/go v1.1.12 go: downloading google.golang.org/grpc v1.47.0 go: downloading github.com/googleapis/gax-go/v2 v2.4.0 go: downloading k8s.io/csi-translation-lib v0.24.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.18 go: downloading k8s.io/apiextensions-apiserver v0.24.0 go: downloading sigs.k8s.io/gateway-api v0.4.1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.5.1 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.18.1 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/go-test/deep v1.0.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/klauspost/compress v1.15.4 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/kr/pretty v0.3.0 go: downloading golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f go: downloading github.com/armon/go-metrics v0.3.10 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.48 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/stretchr/objx v0.4.0 go: downloading google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/docker/docker v20.10.16+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 go: downloading github.com/containerd/containerd v1.6.3 go: downloading oras.land/oras-go v1.1.1 go: downloading github.com/distribution/distribution/v3 v3.0.0-20211118083504-a29a3c99a684 go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.2 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.1.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.3 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/emicklei/go-restful v2.9.5+incompatible go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.1 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 go: downloading github.com/Microsoft/hcsshim v0.9.2 go: downloading github.com/moby/sys/mountinfo v0.5.0 go: downloading github.com/Microsoft/go-winio v0.5.2 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.24.0/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.24.0/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220715-565-g6162eaf6a6" k8s.io/kops/cmd/kops go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0724 17:16:04.904079 23778 addon.go:229] addon test wants to update all nodes I0724 17:16:04.905254 23778 addon.go:229] addon test wants to update worker nodes I0724 17:16:04.905535 23778 addon.go:229] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0724 17:16:04.905831 23778 addon.go:262] installing PKI for "test" I0724 17:16:06.003850 23778 addon.go:262] installing PKI for "test" --- PASS: Test_InstallPKI (1.39s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.428s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.040s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0724 17:16:35.897139 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:35.897391 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:35.897442 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:35.897495 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:35.897544 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:35.897659 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:35.897686 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:35.897706 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:35.897742 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:35.897834 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:35.897868 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:35.982734 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:35.982859 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:35.983547 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.005401 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.005627 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.005669 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.005713 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.005758 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.005793 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.005822 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.005878 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.005904 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.005979 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.006023 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.051661 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.051714 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.051922 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.057946 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.058070 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.058098 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.058122 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.058147 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.058169 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.058186 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.058205 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.058219 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.058270 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.058296 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.093148 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.093212 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.093426 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.100871 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.101028 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.101060 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.101089 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.101140 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.101165 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.101182 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.101204 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.101220 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.101277 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.101308 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.150125 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.150205 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.150538 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.156178 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.156322 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.156352 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.156378 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.156407 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.156432 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.156449 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.156475 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.156492 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.156548 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.156571 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.187425 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.187478 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.187678 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.193636 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.193793 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.193819 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.193846 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.193871 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.193893 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.193909 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.193929 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.193943 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.193992 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.194015 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.240110 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.240185 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.240466 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.35s) === RUN TestCreateClusterOverride I0724 17:16:36.246719 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.246886 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.246917 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.246946 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.246990 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.247047 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.247067 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.247089 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.247105 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.247211 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.247247 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.293899 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.294317 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.294556 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.05s) === RUN TestCreateClusterComplex I0724 17:16:36.300157 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.300299 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.300338 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.300363 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.300390 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.300440 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.300457 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.300478 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.300495 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.300556 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.300583 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.324434 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.324487 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.324682 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.04s) === RUN TestCreateClusterHA I0724 17:16:36.335793 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.336062 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.336117 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.336166 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.336219 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.336304 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.336333 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.336368 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.336397 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.336490 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.336533 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.460706 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.460770 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.460784 29520 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0724 17:16:36.460793 29520 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0724 17:16:36.461050 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.470378 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.470501 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.470527 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.470550 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.470573 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.470593 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.470608 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.470625 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.470637 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.470688 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.470711 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.492223 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.492327 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.492340 29520 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0724 17:16:36.492350 29520 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0724 17:16:36.492651 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.17s) === RUN TestCreateClusterHAGCE I0724 17:16:36.502260 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.502435 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.502466 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.502494 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.502522 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.502576 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.502594 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.502617 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.502635 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.502691 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.502716 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.534041 29520 new_cluster.go:1168] Cloud Provider ID = gce I0724 17:16:36.534135 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.04s) === RUN TestCreateClusterGCE I0724 17:16:36.545348 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.545600 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.545650 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.545695 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.545740 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.545814 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.545842 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.545874 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.545899 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.545979 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.546021 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.624834 29520 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0724 17:16:36.624883 29520 new_cluster.go:1168] Cloud Provider ID = gce I0724 17:16:36.624915 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.09s) === RUN TestCreateClusterHASharedZone I0724 17:16:36.630426 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.630568 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.630593 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.630618 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.630642 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.630682 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.630698 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.630719 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.630734 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.630777 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.630800 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0724 17:16:36.660151 29520 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0724 17:16:36.660190 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.660236 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.660462 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.04s) === RUN TestCreateClusterHASharedZones I0724 17:16:36.667558 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.667681 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.667705 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.667729 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.667752 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.667793 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.667809 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.667827 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.667843 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.667890 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.667914 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0724 17:16:36.714468 29520 new_cluster.go:781] Running with masters in the same AZs; redundancy will be reduced I0724 17:16:36.714553 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.714655 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.714678 29520 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0724 17:16:36.715166 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.06s) === RUN TestCreateClusterPrivate I0724 17:16:36.732298 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.732577 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.732627 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.732671 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.732723 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.732806 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.732834 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.732871 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.732897 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.732991 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.733033 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.812757 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:36.812977 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:36.813008 29520 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0724 17:16:36.814507 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.814580 29520 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.10s) === RUN TestCreateClusterPrivateGCE I0724 17:16:36.827488 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.827747 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.827808 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.827883 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.827967 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.828056 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.828109 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.828172 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.828217 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.828326 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.828383 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.888822 29520 new_cluster.go:425] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0724 17:16:36.888892 29520 new_cluster.go:1168] Cloud Provider ID = gce I0724 17:16:36.888941 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.07s) === RUN TestCreateClusterWithNGWSpecified I0724 17:16:36.895991 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.896139 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.896175 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.896213 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.896256 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.896310 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.896341 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.896377 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.896402 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.896471 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:36.896504 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.124072 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.124131 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:37.124144 29520 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0724 17:16:37.124382 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:37.124419 29520 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.24s) === RUN TestCreateClusterWithINGWSpecified I0724 17:16:37.134202 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.134447 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.134496 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.134550 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.134596 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.134652 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.134695 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.134729 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.134757 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.134843 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.134886 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.214971 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.215023 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:37.215037 29520 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0724 17:16:37.215256 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:37.215283 29520 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.09s) === RUN TestCreateClusterSharedVPC I0724 17:16:37.221511 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.221633 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.221658 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.221681 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.221705 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.221746 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.221762 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.221780 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.221794 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.221835 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.221858 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.262318 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.262358 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.262400 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.262431 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.262448 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.262483 29520 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0724 17:16:37.262675 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.05s) === RUN TestCreateClusterSharedSubnets I0724 17:16:37.270097 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.270408 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.270444 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.270498 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.270537 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.270593 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.270616 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.270642 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.270661 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.270757 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.270793 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.385785 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.385817 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.385872 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.385903 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.385922 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.385946 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.385962 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.386196 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.12s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0724 17:16:37.391480 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.391605 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.391632 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.391655 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.391678 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.391717 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.391732 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.391749 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.391762 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.391803 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.391826 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.422280 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:37.422313 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.422331 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.422379 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.422409 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.422427 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.422449 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.422466 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.422773 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.04s) === RUN TestCreateClusterPrivateSharedSubnets I0724 17:16:37.430049 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.430187 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.430211 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.430236 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.430260 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.430301 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.430338 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.430358 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.430373 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.430415 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.430436 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.519719 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.519749 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.519803 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.519824 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.519849 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.519877 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.519895 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.519918 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:16:37.519933 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:37.520156 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.10s) === RUN TestCreateClusterIPv6 I0724 17:16:37.525506 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.525628 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.525652 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.525673 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.525695 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.525733 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.525749 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.525766 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.525778 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.525819 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.525842 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.576569 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.576625 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:37.576907 29520 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.06s) === RUN TestCreateClusterKarpenter I0724 17:16:37.587202 29520 featureflag.go:162] FeatureFlag "Karpenter"=true I0724 17:16:37.587228 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.587458 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.587503 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.587550 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.587596 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.587689 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.587719 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.587751 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.587777 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.587854 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.587896 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0724 17:16:37.650804 29520 new_cluster.go:1168] Cloud Provider ID = aws I0724 17:16:37.650877 29520 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0724 17:16:37.650895 29520 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0724 17:16:37.650906 29520 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0724 17:16:37.651212 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:37.657896 29520 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.07s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0724 17:16:37.658092 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.658226 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.658253 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.658278 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.658303 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.658345 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.658361 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.658381 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.658396 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.674405 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:37.676070 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:37.676146 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:16:39.206962 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:40.905616 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:16:40.907248 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:40.911438 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:40.911649 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:40.913325 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:40.913361 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:40.913492 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:40.914126 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:40.920180 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:40.921169 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:40.921460 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:40.922745 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:40.922804 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:40.922884 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:40.923294 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:40.964342 29520 target.go:112] Terraform output is in /tmp/test1551192212/out I0724 17:16:40.968013 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:40.968256 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:40.968310 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:40.968370 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:40.968420 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:40.968468 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:40.968502 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:40.968539 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:40.968566 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:40.992060 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:40.994848 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:40.994942 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:16:41.033779 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:41.113074 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:16:41.113584 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:41.113626 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:41.113926 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:41.113949 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:41.114170 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:41.114074 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:16:41.114679 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:41.114719 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:41.114809 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:41.116503 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:41.121704 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:41.121983 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.122230 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.122853 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.127371 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.127532 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:41.127844 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:41.132876 29520 target.go:168] Cloudformation output is in /tmp/test649946749/out --- PASS: TestMinimal (3.48s) === RUN TestMinimal_v1_23 I0724 17:16:41.142715 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:41.142955 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:41.143002 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:41.143054 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:41.143101 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:41.143180 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:41.143209 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:41.143252 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:41.143289 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:41.168550 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0724 17:16:41.171539 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.171629 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.215112 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:41.340649 29520 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0724 17:16:41.341157 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:41.341197 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:41.341324 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:41.341420 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:41.341690 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:41.341883 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:41.343952 29520 executor.go:111] Tasks: 45 done / 90 total; 19 can run I0724 17:16:41.351877 29520 executor.go:111] Tasks: 64 done / 90 total; 24 can run I0724 17:16:41.352454 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.352662 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.353501 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.355468 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.355592 29520 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0724 17:16:41.355867 29520 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0724 17:16:41.392716 29520 target.go:112] Terraform output is in /tmp/test1307012937/out I0724 17:16:41.395186 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:41.395329 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:41.395370 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:41.395401 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:41.395431 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:41.395457 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:41.395475 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:41.395498 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:41.395515 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:41.415217 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:41.417991 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.418089 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:16:41.457382 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:41.538980 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:16:41.539478 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:41.539521 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:41.539488 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:41.539558 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:16:41.539622 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:41.539581 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:41.539657 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:41.539643 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:41.539780 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:16:41.541712 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:41.546747 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:41.547074 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.547277 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.550300 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.552596 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.552695 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:41.552981 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:41.555958 29520 target.go:168] Cloudformation output is in /tmp/test3466193108/out --- PASS: TestMinimal_v1_23 (0.42s) === RUN TestMinimal_v1_24 I0724 17:16:41.566407 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:41.566651 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:41.566707 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:41.566753 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:41.566805 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:41.566881 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:41.566909 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:41.566945 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:41.566995 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:41.592143 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:16:41.594942 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.595024 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.635222 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:41.788202 29520 executor.go:111] Tasks: 0 done / 91 total; 46 can run W0724 17:16:41.788540 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:41.788699 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:41.788723 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:41.788802 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:41.789465 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:41.789672 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:41.793510 29520 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0724 17:16:41.797555 29520 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0724 17:16:41.797925 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.798111 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.798649 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.800977 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.801098 29520 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0724 17:16:41.802052 29520 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0724 17:16:41.852734 29520 target.go:112] Terraform output is in /tmp/test2711817520/out I0724 17:16:41.855188 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:41.855328 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:41.855367 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:41.855400 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:41.855432 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:41.855459 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:41.855476 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:41.855498 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:41.855516 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:41.868669 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:41.870267 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:41.870341 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:16:41.908414 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:42.000676 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:16:42.001045 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:42.001071 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:42.001156 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:42.001165 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:42.001254 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:42.001252 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0724 17:16:42.001275 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:16:42.001318 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:42.001340 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:42.003159 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:42.006290 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:42.006582 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.007275 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.008912 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.010428 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.010512 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:42.010701 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:42.012361 29520 target.go:168] Cloudformation output is in /tmp/test1269739930/out --- PASS: TestMinimal_v1_24 (0.45s) === RUN TestNvidia I0724 17:16:42.018069 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:42.018205 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:42.018237 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:42.018264 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:42.018293 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:42.018339 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:42.018355 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:42.018378 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:42.018395 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:42.032108 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:42.033785 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.033844 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.071875 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:42.158099 29520 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0724 17:16:42.158635 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:42.158703 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:42.158737 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:42.158860 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:42.158914 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:42.159088 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:42.161614 29520 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0724 17:16:42.169511 29520 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0724 17:16:42.169942 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.170152 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.170703 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.172556 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.172682 29520 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0724 17:16:42.172941 29520 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0724 17:16:42.217175 29520 target.go:112] Terraform output is in /tmp/test463199855/out I0724 17:16:42.221142 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:42.221394 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:42.221449 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:42.221501 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:42.221556 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:42.221598 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:42.221625 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:42.221661 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:42.221690 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:42.245790 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:42.248534 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.248628 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.288410 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:42.376607 29520 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0724 17:16:42.376918 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:42.376917 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:42.377077 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:16:42.377107 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:42.377117 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:42.377229 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:16:42.377252 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:42.377134 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:42.377301 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:16:42.381878 29520 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0724 17:16:42.387533 29520 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0724 17:16:42.388187 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.388460 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.389627 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.390546 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.390677 29520 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0724 17:16:42.390930 29520 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0724 17:16:42.394154 29520 target.go:168] Cloudformation output is in /tmp/test4019122936/out --- PASS: TestNvidia (0.39s) === RUN TestMinimalGossip I0724 17:16:42.404035 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:42.404274 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:42.404322 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:42.404366 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:42.404415 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:42.404490 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:42.404518 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:42.404555 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:42.404581 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:42.434051 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.434149 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.472702 29520 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0724 17:16:42.536666 29520 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0724 17:16:42.536872 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:42.537252 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:42.537492 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:42.539896 29520 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0724 17:16:42.548182 29520 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0724 17:16:42.548744 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.548991 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.549842 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.551301 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.551419 29520 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0724 17:16:42.551733 29520 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0724 17:16:42.601488 29520 target.go:112] Terraform output is in /tmp/test2631448335/out --- PASS: TestMinimalGossip (0.20s) === RUN TestMinimalGossipIRSA I0724 17:16:42.605472 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:42.605725 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:42.605782 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:42.605837 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:42.605888 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:42.605978 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:42.606007 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:42.606045 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:42.606073 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:42.633595 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.633715 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.672061 29520 apply_cluster.go:467] Gossip DNS: skipping DNS validation I0724 17:16:42.750386 29520 executor.go:111] Tasks: 0 done / 87 total; 44 can run W0724 17:16:42.750799 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:42.750870 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:42.753163 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:42.756953 29520 executor.go:111] Tasks: 44 done / 87 total; 20 can run I0724 17:16:42.762462 29520 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0724 17:16:42.762768 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.762939 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.763559 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.764099 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:42.764165 29520 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0724 17:16:42.764345 29520 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0724 17:16:42.807685 29520 target.go:112] Terraform output is in /tmp/test3511914112/out --- PASS: TestMinimalGossipIRSA (0.21s) === RUN TestMinimalGCE I0724 17:16:42.811695 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:42.813555 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0724 17:16:43.036638 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:16:43.036777 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:16:43.036956 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0724 17:16:43.037034 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0724 17:16:43.037068 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0724 17:16:43.037114 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0724 17:16:43.037205 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0724 17:16:43.039345 29520 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0724 17:16:43.050331 29520 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0724 17:16:43.055410 29520 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0724 17:16:43.058008 29520 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0724 17:16:43.058238 29520 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0724 17:16:43.082645 29520 target.go:112] Terraform output is in /tmp/test2346989257/out --- PASS: TestMinimalGCE (0.27s) === RUN TestMinimalGCEPrivate I0724 17:16:43.085277 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:43.086530 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0724 17:16:43.240399 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:16:43.240491 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:16:43.240620 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0724 17:16:43.240657 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0724 17:16:43.240682 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0724 17:16:43.240717 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0724 17:16:43.240786 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0724 17:16:43.242588 29520 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0724 17:16:43.245307 29520 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0724 17:16:43.248683 29520 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0724 17:16:43.255277 29520 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0724 17:16:43.256596 29520 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0724 17:16:43.256753 29520 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0724 17:16:43.286767 29520 target.go:112] Terraform output is in /tmp/test1695958130/out --- PASS: TestMinimalGCEPrivate (0.21s) === RUN TestMinimalGCEInternalLoadBalancer I0724 17:16:43.290624 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:43.292515 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0724 17:16:43.462165 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:16:43.462235 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:16:43.462365 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0724 17:16:43.462400 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0724 17:16:43.462425 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0724 17:16:43.462461 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0724 17:16:43.462527 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0724 17:16:43.462571 29520 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0724 17:16:43.464335 29520 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0724 17:16:43.466694 29520 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0724 17:16:43.473616 29520 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0724 17:16:43.473676 29520 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0724 17:16:43.476269 29520 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0724 17:16:43.477635 29520 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0724 17:16:43.478030 29520 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0724 17:16:43.478280 29520 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0724 17:16:43.506936 29520 target.go:112] Terraform output is in /tmp/test2736607671/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.22s) === RUN TestMinimalGCELongClusterName I0724 17:16:43.509094 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:43.510141 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0724 17:16:43.708410 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:16:43.708487 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:16:43.708567 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0724 17:16:43.708590 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0724 17:16:43.708606 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0724 17:16:43.708626 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0724 17:16:43.708674 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0724 17:16:43.709832 29520 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0724 17:16:43.713131 29520 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0724 17:16:43.719343 29520 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0724 17:16:43.723225 29520 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0724 17:16:43.723430 29520 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0724 17:16:43.754582 29520 target.go:112] Terraform output is in /tmp/test1705354840/out --- PASS: TestMinimalGCELongClusterName (0.25s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0724 17:16:43.757024 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:43.758186 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0724 17:16:43.966192 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:16:43.966249 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:16:43.966338 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0724 17:16:43.966362 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0724 17:16:43.966379 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0724 17:16:43.966401 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0724 17:16:43.966447 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0724 17:16:43.966480 29520 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0724 17:16:43.967585 29520 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0724 17:16:43.969445 29520 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0724 17:16:43.973353 29520 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0724 17:16:43.973373 29520 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0724 17:16:43.977424 29520 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0724 17:16:43.977668 29520 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0724 17:16:43.977846 29520 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0724 17:16:43.977989 29520 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0724 17:16:44.010218 29520 target.go:112] Terraform output is in /tmp/test3731130516/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.26s) === RUN TestHA I0724 17:16:44.014247 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:44.014493 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:44.014546 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:44.014598 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:44.014651 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:44.014731 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:44.014760 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:44.014797 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:44.014824 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:44.046201 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:44.049063 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.049154 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.049214 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.049274 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.087635 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:44.174568 29520 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0724 17:16:44.174752 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:44.174982 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:44.175196 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:44.175331 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:44.175418 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:44.175426 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:44.175446 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:44.175540 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:44.178078 29520 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0724 17:16:44.185479 29520 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0724 17:16:44.185815 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.185900 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.186043 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.186232 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.187106 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.187351 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.187947 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.188111 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.188185 29520 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0724 17:16:44.188489 29520 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0724 17:16:44.265209 29520 target.go:112] Terraform output is in /tmp/test602670612/out --- PASS: TestHA (0.26s) === RUN TestHighAvailabilityGCE I0724 17:16:44.269881 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:44.272040 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* W0724 17:16:44.433327 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:16:44.433423 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:16:44.433543 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0724 17:16:44.433579 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0724 17:16:44.433603 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0724 17:16:44.433637 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0724 17:16:44.433720 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0724 17:16:44.433799 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0724 17:16:44.433866 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0724 17:16:44.435993 29520 executor.go:111] Tasks: 0 done / 79 total; 44 can run I0724 17:16:44.438797 29520 executor.go:111] Tasks: 44 done / 79 total; 20 can run I0724 17:16:44.447707 29520 executor.go:111] Tasks: 64 done / 79 total; 7 can run I0724 17:16:44.450415 29520 executor.go:111] Tasks: 71 done / 79 total; 5 can run I0724 17:16:44.451277 29520 executor.go:111] Tasks: 76 done / 79 total; 3 can run I0724 17:16:44.451514 29520 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0724 17:16:44.491294 29520 target.go:112] Terraform output is in /tmp/test2161171687/out --- PASS: TestHighAvailabilityGCE (0.23s) === RUN TestComplex I0724 17:16:44.496387 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:44.496641 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:44.496697 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:44.496834 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:44.496886 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:44.496928 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:44.496959 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:44.497018 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:44.497052 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:44.532128 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:44.536030 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.536139 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.574927 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:44.650371 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0724 17:16:44.650765 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:44.650987 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:44.651095 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:44.651123 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:44.651212 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:44.652944 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:16:44.659945 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:16:44.660206 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.660351 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.661057 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.662703 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.662850 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:16:44.663282 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0724 17:16:44.731775 29520 target.go:112] Terraform output is in /tmp/test1317371299/out I0724 17:16:44.735460 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:44.735689 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:44.735750 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:44.735804 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:44.735859 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:44.735902 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:44.735933 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:44.735969 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:44.735998 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:44.767791 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:44.771412 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.771514 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.810424 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:44.888768 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0724 17:16:44.889341 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:44.889421 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:16:44.889447 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:44.889442 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:44.889509 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:44.894906 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:16:44.901590 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:16:44.902049 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.902200 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.902881 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.903560 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.903636 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:16:44.904088 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0724 17:16:44.908681 29520 target.go:168] Cloudformation output is in /tmp/test3286476858/out I0724 17:16:44.923429 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:44.923703 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:44.923759 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:44.923806 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:44.923857 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:44.923897 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:44.923925 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:44.923960 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:44.923986 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:44.956152 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:44.960085 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.960190 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:44.999109 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:45.102759 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0724 17:16:45.103440 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:45.103669 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:45.103781 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:45.103813 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:45.103916 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:45.106064 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:16:45.115529 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:16:45.115753 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.116398 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.117013 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.118776 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.118955 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:16:45.119380 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0724 17:16:45.187640 29520 target.go:112] Terraform output is in /tmp/test1317926817/out --- PASS: TestComplex (0.70s) === RUN TestCompress I0724 17:16:45.192078 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:45.192330 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:45.192389 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:45.192444 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:45.192497 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:45.192584 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:45.192615 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:45.192655 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:45.192683 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:45.222219 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:45.225147 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.225244 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.264286 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:45.365764 29520 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0724 17:16:45.366698 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:45.366733 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:16:45.366845 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:45.366847 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:45.366882 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:45.368797 29520 executor.go:111] Tasks: 42 done / 84 total; 19 can run I0724 17:16:45.377240 29520 executor.go:111] Tasks: 61 done / 84 total; 21 can run I0724 17:16:45.377903 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.378201 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.381190 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.382819 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.382943 29520 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0724 17:16:45.383274 29520 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0724 17:16:45.436162 29520 target.go:112] Terraform output is in /tmp/test1078779273/out --- PASS: TestCompress (0.25s) === RUN TestExternalPolicies I0724 17:16:45.440126 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:45.440795 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:45.440847 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:45.440905 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:45.440960 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:45.441051 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:45.441081 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:45.441119 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:45.441149 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:45.468594 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:45.472122 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.472224 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.510791 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:45.610218 29520 executor.go:111] Tasks: 0 done / 98 total; 45 can run W0724 17:16:45.610706 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:45.611118 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:45.611121 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:45.611199 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:45.611244 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:45.611384 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:45.613988 29520 executor.go:111] Tasks: 45 done / 98 total; 20 can run I0724 17:16:45.622594 29520 executor.go:111] Tasks: 65 done / 98 total; 30 can run I0724 17:16:45.622953 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.623462 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.624080 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.625984 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.626106 29520 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0724 17:16:45.626521 29520 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0724 17:16:45.691905 29520 target.go:112] Terraform output is in /tmp/test3325999219/out --- PASS: TestExternalPolicies (0.26s) === RUN TestMinimalIPv6 I0724 17:16:45.695926 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:45.696185 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:45.696240 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:45.696291 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:45.696344 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:45.696434 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:45.696464 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:45.696502 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:45.696530 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:45.722102 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:45.725395 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.725491 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.764268 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:45.936763 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0724 17:16:45.937475 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:45.937723 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:45.937915 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:45.938022 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:45.937957 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:45.938102 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:45.943254 29520 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0724 17:16:45.950076 29520 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0724 17:16:45.950742 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.950979 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.952250 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.953330 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:45.953458 29520 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0724 17:16:45.954216 29520 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0724 17:16:45.954621 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:16:46.018808 29520 target.go:112] Terraform output is in /tmp/test464127756/out I0724 17:16:46.023057 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:46.023296 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:46.023362 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:46.023413 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:46.023469 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:46.023513 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:46.023542 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:46.023581 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:46.023654 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:46.049225 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:46.052545 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.052648 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.091668 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:46.259095 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run W0724 17:16:46.259345 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:46.259663 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:46.259733 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:46.259769 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:46.259874 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:46.260202 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:46.260433 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:46.260466 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:46.260656 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:16:46.262811 29520 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0724 17:16:46.270887 29520 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0724 17:16:46.271293 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.271610 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.272376 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.274281 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.274497 29520 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0724 17:16:46.275163 29520 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0724 17:16:46.275387 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:16:46.279423 29520 target.go:168] Cloudformation output is in /tmp/test1241947035/out --- PASS: TestMinimalIPv6 (0.60s) === RUN TestMinimalIPv6Private I0724 17:16:46.291016 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:46.291260 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:46.291312 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:46.291361 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:46.291412 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:46.291495 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:46.291524 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:46.291560 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:46.291586 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:46.324677 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0724 17:16:46.328306 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.328409 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.368008 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:46.551872 29520 executor.go:111] Tasks: 0 done / 122 total; 46 can run I0724 17:16:46.552172 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:46.552313 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:46.552219 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:46.552368 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:46.553133 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:46.553965 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:46.559598 29520 executor.go:111] Tasks: 46 done / 122 total; 23 can run I0724 17:16:46.566155 29520 executor.go:111] Tasks: 69 done / 122 total; 36 can run I0724 17:16:46.566626 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.566957 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.567921 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.569550 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.569676 29520 executor.go:111] Tasks: 105 done / 122 total; 11 can run I0724 17:16:46.570236 29520 executor.go:111] Tasks: 116 done / 122 total; 6 can run I0724 17:16:46.570761 29520 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0724 17:16:46.645239 29520 target.go:112] Terraform output is in /tmp/test2972782287/out --- PASS: TestMinimalIPv6Private (0.36s) === RUN TestMinimalIPv6Calico I0724 17:16:46.649640 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:46.649892 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:46.649949 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:46.650001 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:46.650050 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:46.650139 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:46.650169 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:46.650204 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:46.650231 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:46.679623 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:16:46.683497 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.683606 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:46.727360 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:47.150413 29520 executor.go:111] Tasks: 0 done / 107 total; 47 can run I0724 17:16:47.150703 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:47.151046 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:47.151103 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:47.151266 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:47.151444 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:47.151594 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:47.154107 29520 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0724 17:16:47.159493 29520 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0724 17:16:47.159978 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.160216 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.161472 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.162582 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.162709 29520 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0724 17:16:47.163221 29520 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0724 17:16:47.163539 29520 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0724 17:16:47.225268 29520 target.go:112] Terraform output is in /tmp/test286509688/out I0724 17:16:47.235070 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:47.235357 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:47.235417 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:47.235472 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:47.235529 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:47.235571 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:47.235600 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:47.235639 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:47.235670 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:47.261399 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:16:47.264748 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.264846 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.302761 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:47.728305 29520 executor.go:111] Tasks: 0 done / 107 total; 47 can run I0724 17:16:47.728695 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:47.728895 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:47.728921 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:47.729202 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:47.729099 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } W0724 17:16:47.729247 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:47.729465 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:47.729501 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:47.729639 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:47.732027 29520 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0724 17:16:47.737511 29520 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0724 17:16:47.738004 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.738244 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.741276 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.745507 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.745684 29520 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0724 17:16:47.746083 29520 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0724 17:16:47.746393 29520 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0724 17:16:47.750033 29520 target.go:168] Cloudformation output is in /tmp/test1971146096/out --- PASS: TestMinimalIPv6Calico (1.11s) === RUN TestMinimalIPv6Cilium I0724 17:16:47.761226 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:47.761470 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:47.761525 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:47.761571 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:47.761621 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:47.761697 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:47.761726 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:47.761765 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:47.761795 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:47.786931 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:47.790154 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.790244 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:47.831010 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:48.046867 29520 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0724 17:16:48.047385 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:48.047814 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:48.047852 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:48.047945 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:48.047864 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:48.047994 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:48.050368 29520 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0724 17:16:48.061036 29520 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0724 17:16:48.061439 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.061779 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.063024 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.064094 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.064235 29520 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0724 17:16:48.064686 29520 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0724 17:16:48.065006 29520 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0724 17:16:48.130113 29520 target.go:112] Terraform output is in /tmp/test1401717566/out I0724 17:16:48.134568 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:48.134819 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:48.134879 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:48.134940 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:48.135008 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:48.135067 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:48.135096 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:48.135140 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:48.135174 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:48.160271 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:48.163916 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.164014 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.204104 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:48.420794 29520 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0724 17:16:48.421549 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:48.421574 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:48.421752 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:16:48.421780 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:48.421790 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:48.421938 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:48.421806 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:48.421963 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:48.422023 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:16:48.423713 29520 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0724 17:16:48.433293 29520 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0724 17:16:48.433612 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.433906 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.435112 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.436605 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.436871 29520 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0724 17:16:48.437324 29520 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0724 17:16:48.437625 29520 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0724 17:16:48.441661 29520 target.go:168] Cloudformation output is in /tmp/test2324958001/out --- PASS: TestMinimalIPv6Cilium (0.69s) === RUN TestMinimalWarmPool I0724 17:16:48.452962 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:48.453201 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:48.453255 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:48.453300 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:48.453347 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:48.453427 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:48.453458 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:48.453498 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:48.453530 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:48.482096 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:48.485470 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.485570 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.525815 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:48.724203 29520 executor.go:111] Tasks: 0 done / 88 total; 45 can run I0724 17:16:48.724632 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:48.724663 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:48.724744 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:48.725007 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:48.725110 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:48.725129 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:48.731402 29520 executor.go:111] Tasks: 45 done / 88 total; 19 can run I0724 17:16:48.736909 29520 executor.go:111] Tasks: 64 done / 88 total; 21 can run I0724 17:16:48.737318 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.737535 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.738882 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.740066 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.740183 29520 executor.go:111] Tasks: 85 done / 88 total; 2 can run I0724 17:16:48.740447 29520 executor.go:111] Tasks: 87 done / 88 total; 1 can run I0724 17:16:48.740764 29520 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0724 17:16:48.779717 29520 target.go:112] Terraform output is in /tmp/test1102558765/out --- PASS: TestMinimalWarmPool (0.33s) === RUN TestMinimalEtcd I0724 17:16:48.782262 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:48.782421 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:48.782462 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:48.782529 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:48.782562 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:48.782588 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:48.782606 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:48.782628 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:48.782645 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:48.810228 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:48.813262 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.813358 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.851873 29520 zones.go:93] ListHostedZonesPages { } W0724 17:16:48.938962 29520 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0724 17:16:48.939023 29520 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0724 17:16:48.941345 29520 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0724 17:16:48.941503 29520 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0724 17:16:48.941551 29520 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:48.953866 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:16:48.954271 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:48.954348 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:48.954444 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0724 17:16:48.954491 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:48.954455 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:48.954536 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0724 17:16:48.954305 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:48.954688 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:16:48.954852 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:48.956388 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:48.959552 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:48.959769 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.960132 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.961872 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.965879 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:48.966034 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:48.966288 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:48.969277 29520 target.go:168] Cloudformation output is in /tmp/test2728094293/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0724 17:16:48.979563 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:48.979810 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:48.979860 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:48.979909 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:48.979961 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:48.980039 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:48.980069 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:48.980108 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:48.980135 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:49.003008 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:49.005264 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.005331 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.044330 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:49.145452 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:16:49.145999 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:49.146025 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:49.146031 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:49.146132 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:49.146393 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:49.146528 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:49.148024 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:49.157154 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:49.157652 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.157869 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.158973 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.160051 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.160171 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:49.160489 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:49.211715 29520 target.go:112] Terraform output is in /tmp/test919444826/out I0724 17:16:49.215735 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:49.215985 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:49.216046 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:49.216103 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:49.216155 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:49.216203 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:49.216234 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:49.216277 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:49.216305 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:49.241361 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:49.244313 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.244413 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.284256 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:49.381157 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:16:49.381553 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:49.381602 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:16:49.381727 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:49.381744 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:16:49.381810 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:49.382130 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:49.382278 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:49.382295 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:49.382383 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:16:49.384497 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:49.389628 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:49.390135 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.390344 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.393079 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.395817 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.396000 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:49.396320 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:49.399682 29520 target.go:168] Cloudformation output is in /tmp/test811679901/out --- PASS: TestMinimalGp3 (0.43s) === RUN TestMinimalLongClusterName I0724 17:16:49.409904 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:49.410172 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:49.410226 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:49.410278 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:49.410372 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:49.410475 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:49.410506 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:49.410555 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:49.410592 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:49.434032 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:49.436941 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.437070 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:16:49.475233 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:49.571976 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:16:49.572677 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:49.572731 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:49.572716 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:49.572798 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:49.572837 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:49.573161 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:49.575400 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:49.584086 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:49.584510 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.584722 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.585687 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.587103 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.587229 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:49.587571 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:49.645571 29520 target.go:112] Terraform output is in /tmp/test2718713758/out I0724 17:16:49.649595 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:49.649845 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:49.649903 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:49.649953 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:49.650004 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:49.650046 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:49.650073 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:49.650108 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:49.650135 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:49.673949 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:49.676814 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.676912 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:16:49.715905 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:49.814888 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:16:49.815986 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:49.816028 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:16:49.816149 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:49.816168 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:16:49.816221 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:49.816411 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:49.816559 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:49.816578 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:49.816665 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/this.is.truly.a.really.really.long.cluster-name.minimal.example.com", Value: "owned" },{ Key: "Name", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" },{ Key: "KubernetesCluster", Value: "this.is.truly.a.really.really.long.cluster-name.minimal.example.com" }] }] } I0724 17:16:49.818476 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:16:49.823956 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:16:49.824469 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.824692 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.826188 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.828863 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.830762 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:16:49.831109 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:16:49.834386 29520 target.go:168] Cloudformation output is in /tmp/test3031792416/out --- PASS: TestMinimalLongClusterName (0.43s) === RUN TestExistingIAMCloudformation I0724 17:16:49.844880 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:49.845207 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:49.845278 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:49.845433 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:49.845492 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:49.845531 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:49.845558 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:49.845593 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:49.845618 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:49.869247 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:49.872073 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.872164 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:49.913475 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:50.012525 29520 executor.go:111] Tasks: 0 done / 75 total; 41 can run I0724 17:16:50.013131 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:50.013239 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:50.013345 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:50.013378 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:50.013502 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:50.013539 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:50.013521 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:16:50.013574 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:50.013803 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:50.015940 29520 executor.go:111] Tasks: 41 done / 75 total; 11 can run I0724 17:16:50.021026 29520 executor.go:111] Tasks: 52 done / 75 total; 21 can run I0724 17:16:50.021282 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.021604 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.022355 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.024040 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.024184 29520 executor.go:111] Tasks: 73 done / 75 total; 2 can run I0724 17:16:50.024423 29520 executor.go:111] Tasks: 75 done / 75 total; 0 can run I0724 17:16:50.027466 29520 target.go:168] Cloudformation output is in /tmp/test3841487063/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I0724 17:16:50.038224 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:50.038513 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:50.038570 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:50.038626 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:50.038691 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:50.038793 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:50.038824 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:50.038866 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:50.038906 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:50.072818 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:50.076009 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.076102 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.076172 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.076241 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.114595 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:50.221228 29520 executor.go:111] Tasks: 0 done / 138 total; 49 can run I0724 17:16:50.221575 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:50.221625 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:50.221613 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:50.221743 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:50.221965 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:50.222144 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:50.222247 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:50.222347 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:50.224069 29520 executor.go:111] Tasks: 49 done / 138 total; 26 can run I0724 17:16:50.234321 29520 executor.go:111] Tasks: 75 done / 138 total; 58 can run I0724 17:16:50.234895 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.235203 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.235482 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.235723 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.236262 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.237395 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.239775 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.239922 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.240049 29520 executor.go:111] Tasks: 133 done / 138 total; 5 can run I0724 17:16:50.240645 29520 executor.go:111] Tasks: 138 done / 138 total; 0 can run I0724 17:16:50.333990 29520 target.go:112] Terraform output is in /tmp/test3245315761/out --- PASS: TestExistingSG (0.30s) === RUN TestBastionAdditionalUserData I0724 17:16:50.338824 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:50.339083 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:50.339141 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:50.339192 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:50.339249 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:50.339344 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:50.339376 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:50.339415 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:50.339445 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:50.373346 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:50.376479 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.376579 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.376645 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.415555 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:50.521300 29520 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0724 17:16:50.521929 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:50.521975 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:50.522083 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:50.522218 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:50.522220 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:50.522499 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:50.522503 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:50.524936 29520 executor.go:111] Tasks: 48 done / 119 total; 29 can run I0724 17:16:50.534488 29520 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0724 17:16:50.534804 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.535231 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.535460 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.536170 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.536468 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.538079 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.538209 29520 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0724 17:16:50.538770 29520 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0724 17:16:50.538955 29520 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0724 17:16:50.615359 29520 target.go:112] Terraform output is in /tmp/test3367243500/out --- PASS: TestBastionAdditionalUserData (0.28s) === RUN TestPrivateWeave I0724 17:16:50.619858 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:50.620118 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:50.620179 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:50.620235 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:50.620289 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:50.620402 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:50.620434 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:50.620473 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:50.620502 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:50.648321 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:50.651179 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.651270 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.651332 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.690183 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:50.813125 29520 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0724 17:16:50.813428 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:50.814008 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:50.815833 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:50.815969 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:50.816512 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:50.817574 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:50.818976 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:50.820952 29520 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0724 17:16:50.826775 29520 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0724 17:16:50.827428 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.827593 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.827767 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.827970 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.828108 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.829498 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.829641 29520 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0724 17:16:50.830203 29520 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0724 17:16:50.831006 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:50.906134 29520 target.go:112] Terraform output is in /tmp/test2628580581/out --- PASS: TestPrivateWeave (0.29s) === RUN TestPrivateFlannel I0724 17:16:50.914798 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:50.915102 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:50.915174 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:50.915252 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:50.915338 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:50.915443 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:50.915499 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:50.915561 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:50.915603 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:50.945312 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:50.948339 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.948435 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.948497 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:50.987943 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:51.107715 29520 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0724 17:16:51.108312 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:51.108335 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:51.108378 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:51.108416 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:51.112024 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:51.108345 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:51.112327 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:51.115297 29520 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0724 17:16:51.121148 29520 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0724 17:16:51.121498 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.121728 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.121928 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.122128 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.122673 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.124253 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.124373 29520 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0724 17:16:51.124870 29520 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0724 17:16:51.125068 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:51.201392 29520 target.go:112] Terraform output is in /tmp/test2841685765/out --- PASS: TestPrivateFlannel (0.29s) === RUN TestPrivateCalico I0724 17:16:51.205735 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:51.205985 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:51.206045 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:51.206100 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:51.206154 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:51.206261 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:51.206291 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:51.206329 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:51.206358 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:51.232848 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:16:51.238890 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.238992 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.239057 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.281760 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:51.697486 29520 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0724 17:16:51.697772 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:51.697798 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:51.697892 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:51.698011 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:51.698102 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:51.698581 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:51.698126 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:51.701164 29520 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0724 17:16:51.710972 29520 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0724 17:16:51.711696 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.711890 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.711985 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.712221 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.713142 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.714758 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.714903 29520 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0724 17:16:51.715419 29520 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0724 17:16:51.715591 29520 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0724 17:16:51.789622 29520 target.go:112] Terraform output is in /tmp/test2606203169/out I0724 17:16:51.797468 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:51.797805 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:51.797866 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:51.797924 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:51.797979 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:51.798024 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:51.798052 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:51.798089 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:51.798120 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:51.829403 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:16:51.832869 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.832962 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.833024 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:51.872982 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:52.295983 29520 executor.go:111] Tasks: 0 done / 124 total; 50 can run W0724 17:16:52.296410 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:52.296589 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:52.297698 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:52.297779 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:52.297930 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:16:52.299308 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:52.301852 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:52.302079 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:52.302105 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:52.302240 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:16:52.303883 29520 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0724 17:16:52.309981 29520 executor.go:111] Tasks: 78 done / 124 total; 40 can run I0724 17:16:52.310376 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.310644 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.312559 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.312778 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.316908 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.318458 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.318738 29520 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0724 17:16:52.319211 29520 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0724 17:16:52.319419 29520 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0724 17:16:52.323862 29520 target.go:168] Cloudformation output is in /tmp/test4010342094/out --- PASS: TestPrivateCalico (1.13s) === RUN TestPrivateCilium I0724 17:16:52.336656 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:52.336889 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:52.336940 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:52.337006 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:52.337062 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:52.337141 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:52.337169 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:52.337205 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:52.337235 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:52.364470 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:52.367211 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.367293 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.367350 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.408993 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:52.563459 29520 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0724 17:16:52.563743 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:52.563929 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:52.564046 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:52.564134 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:52.564736 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:52.564780 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:52.564944 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:52.567624 29520 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0724 17:16:52.577147 29520 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0724 17:16:52.577510 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.577647 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.577818 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.578017 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.578891 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.579738 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.579858 29520 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0724 17:16:52.580340 29520 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0724 17:16:52.580518 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:52.657599 29520 target.go:112] Terraform output is in /tmp/test3600901232/out I0724 17:16:52.662053 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:52.662331 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:52.662411 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:52.662490 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:52.662582 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:52.662645 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:52.662700 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:52.662768 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:52.662822 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:52.695776 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:52.699008 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.699110 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.699173 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.740225 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:52.886119 29520 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0724 17:16:52.886386 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:52.886670 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:52.886852 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:52.886824 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:52.886937 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:52.886937 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:52.886952 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:52.886999 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:52.887165 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:52.887061 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0724 17:16:52.888941 29520 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0724 17:16:52.894292 29520 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0724 17:16:52.894538 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.894691 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.894826 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.895091 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.896168 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.897222 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.897362 29520 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0724 17:16:52.897762 29520 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0724 17:16:52.897923 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:52.901989 29520 target.go:168] Cloudformation output is in /tmp/test647067674/out --- PASS: TestPrivateCilium (0.58s) === RUN TestPrivateCilium2 I0724 17:16:52.918845 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:52.919210 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:52.919295 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:52.919383 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:52.919472 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:52.919577 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:52.919608 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:52.919644 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:52.919673 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:52.947715 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0724 17:16:52.951013 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.951099 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.951160 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:52.992325 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:53.129466 29520 executor.go:111] Tasks: 0 done / 117 total; 47 can run W0724 17:16:53.129748 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:53.129881 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:53.130769 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:53.132666 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:53.132709 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:53.132843 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:53.133708 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:53.136862 29520 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0724 17:16:53.143163 29520 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0724 17:16:53.143467 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.143929 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.144175 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.144356 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.145195 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.146568 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.146688 29520 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0724 17:16:53.148331 29520 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0724 17:16:53.148717 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:53.223941 29520 target.go:112] Terraform output is in /tmp/test4095684732/out I0724 17:16:53.228342 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:53.228591 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:53.228654 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:53.228709 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:53.228766 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:53.228810 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:53.228838 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:53.228876 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:53.228910 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:53.260504 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0724 17:16:53.264565 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.264665 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.264735 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.303115 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:53.443398 29520 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0724 17:16:53.444064 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:53.444205 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0724 17:16:53.444235 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:53.444223 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:53.444100 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:16:53.444106 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:53.444442 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:53.444567 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:53.444318 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" },{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" }] }] } W0724 17:16:53.446675 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:53.451404 29520 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0724 17:16:53.458133 29520 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0724 17:16:53.458560 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.458844 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.459145 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.459339 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.460789 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.461178 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.461331 29520 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0724 17:16:53.461691 29520 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0724 17:16:53.461852 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:53.466186 29520 target.go:168] Cloudformation output is in /tmp/test3657497092/out --- PASS: TestPrivateCilium2 (0.56s) === RUN TestPrivateCiliumAdvanced I0724 17:16:53.481787 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:53.482038 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:53.482091 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:53.482138 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:53.482188 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:53.482270 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:53.482299 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:53.482336 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:53.482364 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:53.515542 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:53.518958 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.519053 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.519118 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.557833 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:53.712421 29520 executor.go:111] Tasks: 0 done / 123 total; 53 can run W0724 17:16:53.712800 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:53.713252 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:53.713694 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:53.714234 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:53.715590 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:53.715630 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:53.715759 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:53.720296 29520 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0724 17:16:53.727197 29520 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0724 17:16:53.727555 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.727827 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.728080 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.728335 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.728816 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.730510 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.730631 29520 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0724 17:16:53.731149 29520 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0724 17:16:53.731340 29520 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0724 17:16:53.811299 29520 target.go:112] Terraform output is in /tmp/test362836938/out I0724 17:16:53.816508 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:53.816780 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:53.816852 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:53.816930 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:53.817039 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:53.817102 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:53.817155 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:53.817219 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:53.817266 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:53.850125 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:53.856162 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.856277 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.856349 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:53.895629 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:54.038035 29520 executor.go:111] Tasks: 0 done / 123 total; 53 can run W0724 17:16:54.038320 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:54.038334 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:54.038485 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key W0724 17:16:54.038601 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:54.039487 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:54.040815 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:54.040876 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:54.041024 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:54.038512 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:54.043430 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0724 17:16:54.045948 29520 executor.go:111] Tasks: 53 done / 123 total; 28 can run I0724 17:16:54.052909 29520 executor.go:111] Tasks: 81 done / 123 total; 36 can run I0724 17:16:54.053162 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.053457 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.053676 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.054516 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.055574 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.059149 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.059316 29520 executor.go:111] Tasks: 117 done / 123 total; 5 can run I0724 17:16:54.059745 29520 executor.go:111] Tasks: 122 done / 123 total; 1 can run I0724 17:16:54.059904 29520 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0724 17:16:54.064148 29520 target.go:168] Cloudformation output is in /tmp/test4242589314/out --- PASS: TestPrivateCiliumAdvanced (0.60s) === RUN TestPrivateCanal I0724 17:16:54.078269 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:54.078527 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:54.078579 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:54.078626 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:54.078678 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:54.078758 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:54.078788 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:54.078822 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:54.078850 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:54.106240 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0724 17:16:54.109008 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.109092 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.109146 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.152644 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:54.544251 29520 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0724 17:16:54.544656 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:54.544773 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:54.544910 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:54.545072 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:54.545264 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:54.545298 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:54.545396 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:54.547838 29520 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0724 17:16:54.557562 29520 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0724 17:16:54.558022 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.558140 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.558310 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.558526 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.559481 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.560696 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:54.560838 29520 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0724 17:16:54.561322 29520 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0724 17:16:54.561516 29520 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0724 17:16:54.634376 29520 target.go:112] Terraform output is in /tmp/test2263137942/out --- PASS: TestPrivateCanal (0.56s) === RUN TestPrivateKopeio I0724 17:16:54.639583 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:54.639845 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:54.639901 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:54.639953 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:54.640005 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:54.640097 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:54.640126 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:54.640163 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:54.640195 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:54.673793 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:54.676927 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.677054 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.677122 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.717474 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:54.833150 29520 executor.go:111] Tasks: 0 done / 123 total; 47 can run W0724 17:16:54.834254 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:54.834247 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:54.834300 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:16:54.834340 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:54.834375 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:54.834344 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:54.834506 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:54.838196 29520 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0724 17:16:54.846144 29520 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0724 17:16:54.846672 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.846865 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.847072 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.847167 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.848185 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.848976 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.849105 29520 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0724 17:16:54.849774 29520 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0724 17:16:54.929704 29520 target.go:112] Terraform output is in /tmp/test2146796111/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I0724 17:16:54.934082 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:54.934347 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:54.934408 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:54.934459 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:54.934518 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:54.934612 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:54.934642 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:54.934681 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:54.934712 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:54.965076 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:54.967996 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.968078 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:54.968136 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.007573 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:55.109667 29520 executor.go:111] Tasks: 0 done / 105 total; 45 can run I0724 17:16:55.110274 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:55.110329 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:55.113202 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:55.113563 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:55.113567 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:55.116234 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:55.116310 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:55.118103 29520 executor.go:111] Tasks: 45 done / 105 total; 25 can run I0724 17:16:55.123599 29520 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0724 17:16:55.123839 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.124042 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.124392 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.124688 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.125499 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.127122 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.127264 29520 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0724 17:16:55.127806 29520 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0724 17:16:55.198186 29520 target.go:112] Terraform output is in /tmp/test4176923351/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I0724 17:16:55.202264 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:55.202521 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:55.202582 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:55.202633 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:55.202687 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:55.202777 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:55.202807 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:55.202842 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:55.202872 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:55.231242 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:55.234087 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.234173 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.234228 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.273269 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:55.375811 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run W0724 17:16:55.376953 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:55.377085 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:55.377132 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:55.377298 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:55.377430 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:55.377527 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:55.377588 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:55.382902 29520 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0724 17:16:55.383560 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:55.388756 29520 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0724 17:16:55.389214 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.389494 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.389721 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.389837 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.390922 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.391833 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.391954 29520 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0724 17:16:55.392425 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:16:55.461752 29520 target.go:112] Terraform output is in /tmp/test2523542220/out --- PASS: TestPrivateSharedSubnet (0.26s) === RUN TestPrivateSharedIP I0724 17:16:55.465949 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:55.466204 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:55.466258 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:55.466310 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:55.466363 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:55.466451 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:55.466480 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:55.466516 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:55.466545 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:55.494720 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:55.500806 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.500925 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.500998 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.542236 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:55.650804 29520 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0724 17:16:55.651237 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:55.651408 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:55.651552 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:55.651905 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:55.651999 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:55.652026 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:55.652102 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:55.658203 29520 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0724 17:16:55.658592 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:55.663578 29520 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0724 17:16:55.664273 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.664511 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.664778 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.664900 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.665648 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.667187 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.667327 29520 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0724 17:16:55.668012 29520 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0724 17:16:55.668298 29520 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0724 17:16:55.741683 29520 target.go:112] Terraform output is in /tmp/test1182188518/out I0724 17:16:55.745836 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:55.746074 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:55.746131 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:55.746182 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:55.746234 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:55.746278 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:55.746306 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:55.746342 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:55.746375 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:55.774362 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:55.777413 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.777505 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.777567 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.816820 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:16:55.918805 29520 executor.go:111] Tasks: 0 done / 113 total; 45 can run W0724 17:16:55.919468 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:55.919585 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:16:55.919684 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:16:55.919666 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:55.919871 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:16:55.919932 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:16:55.919907 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:55.919970 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:55.920100 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:16:55.920080 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:16:55.922347 29520 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0724 17:16:55.922645 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:55.931937 29520 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0724 17:16:55.932245 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.932451 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.932617 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.932795 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.933529 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.934569 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:16:55.934729 29520 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0724 17:16:55.935166 29520 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0724 17:16:55.935319 29520 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0724 17:16:55.939395 29520 target.go:168] Cloudformation output is in /tmp/test535975446/out --- PASS: TestPrivateSharedIP (0.49s) === RUN TestPrivateDns1 I0724 17:16:55.953281 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:55.953526 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:55.953582 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:55.953633 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:55.953711 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:55.953797 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:55.953828 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:55.953867 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:55.953897 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:55.988353 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:55.988449 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:55.988517 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.148907 29520 executor.go:111] Tasks: 0 done / 117 total; 46 can run W0724 17:16:56.149197 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:56.149663 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:56.149960 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:56.150012 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:56.151705 29520 executor.go:111] Tasks: 46 done / 117 total; 24 can run I0724 17:16:56.152005 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0724 17:16:56.152044 29520 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0724 17:16:56.152125 29520 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0724 17:16:56.152141 29520 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0724 17:16:56.152431 29520 executor.go:111] Tasks: 70 done / 117 total; 35 can run I0724 17:16:56.153105 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.153324 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.155986 29520 executor.go:111] Tasks: 105 done / 117 total; 5 can run W0724 17:16:56.156086 29520 api.go:53] PageSize not implemented W0724 17:16:56.156311 29520 api.go:53] PageSize not implemented W0724 17:16:56.156115 29520 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0724 17:16:56.156344 29520 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0724 17:16:56.166124 29520 executor.go:111] Tasks: 110 done / 117 total; 5 can run I0724 17:16:56.166305 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.166497 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.167737 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.168951 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.169070 29520 executor.go:111] Tasks: 115 done / 117 total; 2 can run I0724 17:16:56.169370 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:16:56.251425 29520 target.go:112] Terraform output is in /tmp/test140075786/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I0724 17:16:56.255782 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:56.256036 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:56.256095 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:56.256150 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:56.256202 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:56.256289 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:56.256318 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:56.256355 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:56.256384 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:56.286537 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.286622 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.286680 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.420191 29520 executor.go:111] Tasks: 0 done / 113 total; 44 can run W0724 17:16:56.420705 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:56.420875 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:56.420938 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:56.421216 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:56.423785 29520 executor.go:111] Tasks: 44 done / 113 total; 22 can run I0724 17:16:56.424232 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:16:56.424359 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0724 17:16:56.424399 29520 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0724 17:16:56.424480 29520 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0724 17:16:56.424500 29520 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0724 17:16:56.424555 29520 executor.go:111] Tasks: 66 done / 113 total; 35 can run I0724 17:16:56.426719 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.427193 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.432202 29520 executor.go:111] Tasks: 101 done / 113 total; 5 can run W0724 17:16:56.432365 29520 api.go:53] PageSize not implemented W0724 17:16:56.432454 29520 api.go:53] PageSize not implemented W0724 17:16:56.432402 29520 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0724 17:16:56.432483 29520 bootstrapscript.go:89] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"ResourceBasedNaming":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0724 17:16:56.439504 29520 executor.go:111] Tasks: 106 done / 113 total; 5 can run I0724 17:16:56.439765 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.439971 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.441366 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.442698 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.442819 29520 executor.go:111] Tasks: 111 done / 113 total; 2 can run I0724 17:16:56.443139 29520 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0724 17:16:56.516448 29520 target.go:112] Terraform output is in /tmp/test1177442503/out --- PASS: TestPrivateDns2 (0.26s) === RUN TestDiscoveryFeatureGate I0724 17:16:56.520661 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:56.520915 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:56.520976 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:56.521032 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:56.521085 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:56.521170 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:56.521200 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:56.521236 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:56.521264 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:56.550967 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0724 17:16:56.554206 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.554311 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.596046 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:56.707777 29520 executor.go:111] Tasks: 0 done / 90 total; 46 can run W0724 17:16:56.708030 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:56.708528 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:56.708756 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:56.708794 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:56.708959 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:56.709078 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:56.710643 29520 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0724 17:16:56.719492 29520 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0724 17:16:56.719831 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.720096 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.721356 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.722229 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.722344 29520 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0724 17:16:56.722687 29520 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0724 17:16:56.777739 29520 target.go:112] Terraform output is in /tmp/test2788980394/out --- PASS: TestDiscoveryFeatureGate (0.26s) === RUN TestVFSServiceAccountIssuerDiscovery I0724 17:16:56.781975 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:56.782213 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:56.782264 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:56.782312 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:56.782363 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:56.782448 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:56.782480 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:56.782515 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:56.782543 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:56.806899 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:56.809787 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.809878 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.848444 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:56.949357 29520 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0724 17:16:56.949686 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:56.950472 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:56.950516 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:56.950614 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:16:56.950482 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:56.950625 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:56.952630 29520 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0724 17:16:56.961986 29520 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0724 17:16:56.962467 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.962719 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.963575 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.964936 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:56.965056 29520 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0724 17:16:56.965349 29520 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0724 17:16:57.018353 29520 target.go:112] Terraform output is in /tmp/test4253349551/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.24s) === RUN TestAWSLBController I0724 17:16:57.022596 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:57.022863 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:57.022923 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:57.022976 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:57.023033 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:57.023127 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:57.023156 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:57.023194 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:57.023224 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:57.048998 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:57.052104 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.052201 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.092532 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:57.713562 29520 executor.go:111] Tasks: 0 done / 94 total; 49 can run W0724 17:16:57.714064 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:57.714273 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:57.714668 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:57.714859 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:57.714902 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:57.715013 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:57.720793 29520 executor.go:111] Tasks: 49 done / 94 total; 22 can run I0724 17:16:57.728697 29520 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0724 17:16:57.729060 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.729255 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.730163 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.731617 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.731741 29520 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0724 17:16:57.732065 29520 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0724 17:16:57.789073 29520 target.go:112] Terraform output is in /tmp/test3513966196/out --- PASS: TestAWSLBController (0.77s) === RUN TestManyAddons I0724 17:16:57.796555 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:57.796819 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:57.796880 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:57.796931 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:57.797023 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:57.797124 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:57.797155 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:57.797190 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:57.797223 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:57.824039 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:57.827339 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.827425 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:57.867462 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:58.663495 29520 executor.go:111] Tasks: 0 done / 93 total; 50 can run W0724 17:16:58.663951 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:16:58.664136 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:58.664580 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:58.664614 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:58.664707 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:58.664644 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:16:58.671861 29520 executor.go:111] Tasks: 50 done / 93 total; 19 can run I0724 17:16:58.677583 29520 executor.go:111] Tasks: 69 done / 93 total; 22 can run I0724 17:16:58.677855 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:58.678249 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:58.679467 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:58.680503 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:58.680622 29520 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0724 17:16:58.680921 29520 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0724 17:16:58.734365 29520 target.go:112] Terraform output is in /tmp/test2500410450/out --- PASS: TestManyAddons (0.95s) === RUN TestManyAddonsCCMIRSA I0724 17:16:58.746397 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:58.746671 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:58.746733 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:58.746790 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:58.746841 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:58.746937 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:58.746988 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:58.747051 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:58.747077 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:58.775465 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:16:58.779102 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:58.779208 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:58.820081 29520 zones.go:93] ListHostedZonesPages { } I0724 17:16:59.656192 29520 executor.go:111] Tasks: 0 done / 109 total; 59 can run W0724 17:16:59.656554 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:59.656763 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:16:59.656799 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:16:59.656924 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:16:59.656860 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:16:59.657170 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:16:59.666180 29520 executor.go:111] Tasks: 59 done / 109 total; 26 can run I0724 17:16:59.676413 29520 executor.go:111] Tasks: 85 done / 109 total; 22 can run I0724 17:16:59.676654 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:59.676958 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:59.678243 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:59.679889 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:59.680022 29520 executor.go:111] Tasks: 107 done / 109 total; 2 can run I0724 17:16:59.680345 29520 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0724 17:16:59.747077 29520 target.go:112] Terraform output is in /tmp/test1625378476/out --- PASS: TestManyAddonsCCMIRSA (1.01s) === RUN TestManyAddonsCCMIRSA23 I0724 17:16:59.756139 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:59.756409 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:59.756469 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:59.756599 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:59.756654 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:59.756701 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:59.756732 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:59.756771 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:59.756801 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:59.789155 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* I0724 17:16:59.792792 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:59.792901 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:16:59.835455 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:00.710066 29520 executor.go:111] Tasks: 0 done / 110 total; 60 can run W0724 17:17:00.710839 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:00.710970 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:00.711193 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:00.711453 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:00.711498 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:00.711633 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:00.720009 29520 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0724 17:17:00.725904 29520 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0724 17:17:00.726398 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:00.726624 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:00.727877 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:00.729010 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:00.729138 29520 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0724 17:17:00.729458 29520 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0724 17:17:00.800767 29520 target.go:112] Terraform output is in /tmp/test765038/out --- PASS: TestManyAddonsCCMIRSA23 (1.05s) === RUN TestManyAddonsCCMIRSA24 I0724 17:17:00.810531 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:00.810796 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:00.810858 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:00.810912 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:00.810965 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:00.811057 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:00.811088 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:00.811125 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:00.811156 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:00.838322 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:17:00.841411 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:00.841502 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:00.885177 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:01.791949 29520 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0724 17:17:01.792533 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:01.792653 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:01.792896 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:01.793126 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:01.793162 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:01.793302 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:01.799222 29520 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0724 17:17:01.806731 29520 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0724 17:17:01.807276 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:01.807531 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:01.808469 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:01.810154 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:01.810330 29520 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0724 17:17:01.810685 29520 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0724 17:17:01.878217 29520 target.go:112] Terraform output is in /tmp/test796465374/out --- PASS: TestManyAddonsCCMIRSA24 (1.08s) === RUN TestCCM I0724 17:17:01.887861 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:01.888125 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:01.888185 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:01.888237 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:01.888290 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:01.888378 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:01.888408 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:01.888445 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:01.888474 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:01.914517 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:01.917759 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:01.917848 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:01.960040 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:02.763675 29520 executor.go:111] Tasks: 0 done / 94 total; 51 can run I0724 17:17:02.764182 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:02.764219 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:02.764356 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:02.764735 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:02.764766 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:02.764873 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:02.768485 29520 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0724 17:17:02.778463 29520 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0724 17:17:02.778950 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:02.779184 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:02.780509 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:02.781642 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:02.781764 29520 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0724 17:17:02.782315 29520 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0724 17:17:02.834919 29520 target.go:112] Terraform output is in /tmp/test3850680439/out --- PASS: TestCCM (0.96s) === RUN TestExternalDNS I0724 17:17:02.846614 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:02.846897 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:02.846960 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:02.847042 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:02.847099 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:02.847199 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:02.847229 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:02.847267 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:02.847297 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:02.872315 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:02.875238 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:02.875337 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:02.914331 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:03.012046 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:17:03.012680 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:03.012889 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:17:03.012928 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:03.012919 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:03.013016 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:03.013113 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:03.015039 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:03.020699 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:03.021141 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.021332 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.023544 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.027166 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.027312 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:03.027684 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:03.076817 29520 target.go:112] Terraform output is in /tmp/test4223546161/out I0724 17:17:03.080685 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:03.080946 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:03.081006 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:03.081068 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:03.081134 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:03.081192 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:03.081221 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:03.081270 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:03.081301 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:03.108764 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:03.112013 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.112113 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.151350 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:03.252905 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:17:03.253367 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:03.253808 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:03.253954 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:17:03.253960 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:03.254094 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:03.254116 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:03.254001 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:03.254212 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:17:03.254202 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0724 17:17:03.256245 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:03.261944 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:03.262133 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.262421 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.263513 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.264382 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.264523 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:03.264811 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:03.267706 29520 target.go:168] Cloudformation output is in /tmp/test3094138284/out --- PASS: TestExternalDNS (0.43s) === RUN TestExternalDNSIRSA I0724 17:17:03.278219 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:03.278564 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:03.278651 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:03.278732 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:03.278823 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:03.278937 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:03.279010 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:03.279075 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:03.279113 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:03.307001 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:03.309966 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.310061 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.349210 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:03.466598 29520 executor.go:111] Tasks: 0 done / 90 total; 46 can run W0724 17:17:03.467273 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:03.467441 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:03.467490 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:03.467617 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:03.467746 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:03.467769 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:03.469528 29520 executor.go:111] Tasks: 46 done / 90 total; 21 can run I0724 17:17:03.478884 29520 executor.go:111] Tasks: 67 done / 90 total; 21 can run I0724 17:17:03.479244 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.479385 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.480606 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.481557 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.481740 29520 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0724 17:17:03.482227 29520 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0724 17:17:03.538089 29520 target.go:112] Terraform output is in /tmp/test1995236677/out --- PASS: TestExternalDNSIRSA (0.26s) === RUN TestKarpenter I0724 17:17:03.542724 29520 featureflag.go:162] FeatureFlag "Karpenter"=true I0724 17:17:03.542760 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:03.543007 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:03.543060 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:03.543110 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:03.543159 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:03.543246 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:03.543276 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:03.543318 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:03.543347 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:03.574224 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:03.577136 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.577214 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.577271 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.617572 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:03.712549 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.712699 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.712812 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.804511 29520 executor.go:111] Tasks: 0 done / 101 total; 50 can run I0724 17:17:03.804853 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:03.804910 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:03.805048 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:03.804982 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:03.805114 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:03.806683 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:03.807232 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:03.807994 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:03.812510 29520 executor.go:111] Tasks: 50 done / 101 total; 24 can run I0724 17:17:03.818694 29520 executor.go:111] Tasks: 74 done / 101 total; 25 can run I0724 17:17:03.819137 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.819316 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.819434 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.819654 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.820293 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.820446 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.820586 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.822156 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.822284 29520 executor.go:111] Tasks: 99 done / 101 total; 2 can run I0724 17:17:03.822612 29520 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0724 17:17:03.894659 29520 target.go:112] Terraform output is in /tmp/test4289365307/out I0724 17:17:03.900086 29520 featureflag.go:162] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.36s) === RUN TestSharedSubnet I0724 17:17:03.900219 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:03.900495 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:03.900554 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:03.900608 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:03.900662 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:03.900756 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:03.900786 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:03.900825 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:03.900855 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:03.925322 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:03.928172 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.928262 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:03.969202 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:04.068115 29520 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0724 17:17:04.068683 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:04.068729 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:04.068837 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:04.068953 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:04.068969 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:04.070483 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:04.075485 29520 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0724 17:17:04.076228 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:04.081166 29520 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0724 17:17:04.081589 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.081739 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.082878 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.084143 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.084257 29520 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0724 17:17:04.084572 29520 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0724 17:17:04.130818 29520 target.go:112] Terraform output is in /tmp/test835221444/out --- PASS: TestSharedSubnet (0.23s) === RUN TestSharedVPC I0724 17:17:04.134758 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:04.135008 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:04.135058 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:04.135106 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:04.135154 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:04.135241 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:04.135270 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:04.135305 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:04.135332 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:04.159274 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:04.162111 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.162204 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.204365 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:04.303552 29520 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0724 17:17:04.304163 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0724 17:17:04.304186 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:04.304213 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:04.304336 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:04.304320 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:04.304467 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:04.307349 29520 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0724 17:17:04.307710 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:04.317243 29520 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0724 17:17:04.317738 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.318004 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.319033 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.320693 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.320811 29520 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0724 17:17:04.321159 29520 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0724 17:17:04.367055 29520 target.go:112] Terraform output is in /tmp/test2372632474/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I0724 17:17:04.372164 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:04.372560 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:04.372664 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:04.372755 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:04.372858 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:04.373043 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:04.373110 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:04.373276 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:04.374632 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:04.407487 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:04.410744 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.410847 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.410909 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.410972 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.450963 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:04.550230 29520 executor.go:111] Tasks: 0 done / 93 total; 47 can run W0724 17:17:04.551028 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:04.551062 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:04.551139 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:04.551254 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:04.551304 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:04.551285 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:04.551507 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:04.551456 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:04.553791 29520 executor.go:111] Tasks: 47 done / 93 total; 15 can run I0724 17:17:04.564066 29520 executor.go:111] Tasks: 62 done / 93 total; 27 can run I0724 17:17:04.564499 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.564649 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.564814 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.565101 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.565966 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.567295 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.567528 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.567785 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:04.567901 29520 executor.go:111] Tasks: 89 done / 93 total; 4 can run I0724 17:17:04.568324 29520 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0724 17:17:04.639924 29520 target.go:112] Terraform output is in /tmp/test492622616/out --- PASS: TestExistingIAM (0.27s) === RUN TestPhaseNetwork I0724 17:17:04.646496 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:04.646797 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:04.646855 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:04.646905 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:04.646962 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:04.647029 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:04.647058 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:04.647102 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:04.647132 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:04.675320 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:04.678410 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.678501 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.678564 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.717522 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:04.811757 29520 executor.go:111] Tasks: 0 done / 116 total; 46 can run I0724 17:17:04.812191 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:04.812324 29520 executor.go:111] Tasks: 46 done / 116 total; 28 can run I0724 17:17:04.812785 29520 executor.go:111] Tasks: 74 done / 116 total; 36 can run I0724 17:17:04.813142 29520 executor.go:111] Tasks: 110 done / 116 total; 5 can run I0724 17:17:04.813373 29520 executor.go:111] Tasks: 115 done / 116 total; 1 can run I0724 17:17:04.813528 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0724 17:17:04.822524 29520 target.go:112] Terraform output is in /tmp/test1226029939/out --- PASS: TestPhaseNetwork (0.18s) === RUN TestExternalLoadBalancer I0724 17:17:04.823313 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:04.823555 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:04.823609 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:04.823682 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:04.823735 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:04.823824 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:04.823853 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:04.823890 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:04.823919 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:04.852927 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:04.856353 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.856492 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:04.895237 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:04.999032 29520 executor.go:111] Tasks: 0 done / 92 total; 50 can run I0724 17:17:04.999729 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:04.999774 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:04.999873 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:04.999993 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:05.000018 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.000001 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:05.002313 29520 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0724 17:17:05.007705 29520 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0724 17:17:05.008231 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.008546 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.009653 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.010505 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.010622 29520 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0724 17:17:05.010964 29520 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0724 17:17:05.064146 29520 target.go:112] Terraform output is in /tmp/test2719066073/out I0724 17:17:05.068348 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:05.068585 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:05.068644 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:05.068701 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:05.068759 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:05.068801 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:05.068829 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:05.068868 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:05.068895 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:05.097173 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:05.100125 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.100241 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.143497 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:05.247031 29520 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0724 17:17:05.247288 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:05.247387 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:05.247597 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:05.247743 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:05.247762 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:05.247860 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:05.248118 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:17:05.248150 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:05.248222 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:17:05.250488 29520 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0724 17:17:05.255688 29520 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0724 17:17:05.256126 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.256383 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.257447 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.258780 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.258915 29520 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0724 17:17:05.259182 29520 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0724 17:17:05.262398 29520 target.go:168] Cloudformation output is in /tmp/test1559512828/out --- PASS: TestExternalLoadBalancer (0.45s) === RUN TestPhaseIAM integration_test.go:737: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:745: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0724 17:17:05.272230 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:05.272467 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:05.272515 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:05.272560 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:05.272609 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:05.272686 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:05.272716 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:05.272751 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:05.272779 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:05.310331 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:05.313558 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.313687 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.313758 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.313817 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.313869 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.313948 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.314007 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.354662 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:05.461572 29520 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0724 17:17:05.462277 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.462357 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:05.462457 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.462490 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:05.462514 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:17:05.462610 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.462632 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:05.462618 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.464532 29520 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0724 17:17:05.474569 29520 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0724 17:17:05.474974 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.475301 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.475518 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.475780 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.476622 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.477721 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.478038 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.478486 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.478607 29520 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0724 17:17:05.479061 29520 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0724 17:17:05.558119 29520 target.go:112] Terraform output is in /tmp/test2056822120/out I0724 17:17:05.562699 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:05.562930 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:05.562989 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:05.563044 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:05.563097 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:05.563138 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:05.563166 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:05.563203 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:05.563235 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:05.597085 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:05.600351 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.600459 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.600529 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.600593 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.600650 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.600701 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.600753 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.641088 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:05.738799 29520 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0724 17:17:05.739246 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:05.739335 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:17:05.739446 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:05.739371 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:05.739519 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:17:05.739690 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:05.739774 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:05.739863 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:05.739871 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:05.739913 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:05.740065 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0724 17:17:05.742290 29520 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0724 17:17:05.752077 29520 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0724 17:17:05.752497 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.752741 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.753000 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.753288 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.753730 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.755083 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.755351 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.755578 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.755717 29520 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0724 17:17:05.756089 29520 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0724 17:17:05.761567 29520 target.go:168] Cloudformation output is in /tmp/test485873618/out --- PASS: TestMixedInstancesASG (0.51s) === RUN TestMixedInstancesSpotASG I0724 17:17:05.781516 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:05.781802 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:05.781859 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:05.781911 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:05.781965 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:05.782049 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:05.782078 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:05.782115 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:05.782144 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:05.815033 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:05.818634 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.818727 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.818788 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.818849 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.818900 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.818948 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.818995 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.858557 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:05.961650 29520 executor.go:111] Tasks: 0 done / 103 total; 49 can run I0724 17:17:05.962116 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:05.962148 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.962444 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:05.962473 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:05.962568 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:05.962445 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:05.962614 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:05.962678 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:05.964652 29520 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0724 17:17:05.970139 29520 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0724 17:17:05.972094 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.972536 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.973154 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.973445 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.976712 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.977741 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.977923 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.978502 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:05.978633 29520 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0724 17:17:05.979056 29520 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0724 17:17:06.056233 29520 target.go:112] Terraform output is in /tmp/test2047285937/out I0724 17:17:06.060697 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:06.060930 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:06.060993 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:06.061047 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:06.061101 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:06.061144 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:06.061173 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:06.061213 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:06.061242 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:06.099217 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:06.102402 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.102502 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.102564 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.102627 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.102679 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.102726 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.102775 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.141946 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:06.246044 29520 executor.go:111] Tasks: 0 done / 103 total; 49 can run W0724 17:17:06.246652 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:06.246665 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:06.246800 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:06.246865 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:06.247039 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:06.247074 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:06.247113 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:17:06.247103 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:06.247146 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:06.247269 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:17:06.247265 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" },{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" }] }] } I0724 17:17:06.248826 29520 executor.go:111] Tasks: 49 done / 103 total; 23 can run I0724 17:17:06.258338 29520 executor.go:111] Tasks: 72 done / 103 total; 27 can run I0724 17:17:06.258570 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.258755 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.258943 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.259070 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.260092 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.260984 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.261059 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.261387 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.261521 29520 executor.go:111] Tasks: 99 done / 103 total; 4 can run I0724 17:17:06.261904 29520 executor.go:111] Tasks: 103 done / 103 total; 0 can run I0724 17:17:06.266967 29520 target.go:168] Cloudformation output is in /tmp/test956225625/out --- PASS: TestMixedInstancesSpotASG (0.51s) === RUN TestContainerd I0724 17:17:06.286856 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:06.287144 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:06.287214 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:06.287336 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:06.287421 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:06.287481 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:06.287536 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:06.287598 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:06.287641 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:06.312169 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:06.315025 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.315124 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.362461 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:06.463917 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:17:06.464334 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:06.464650 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:06.464834 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:17:06.464868 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:06.464925 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:06.464960 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:17:06.465059 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:06.465102 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:06.465210 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0724 17:17:06.467150 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:06.472241 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:06.472437 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.472820 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.473886 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.474818 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.474966 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:06.475239 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:06.478168 29520 target.go:168] Cloudformation output is in /tmp/test2452605544/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I0724 17:17:06.491380 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:06.491636 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:06.491691 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:06.491783 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:06.491834 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:06.491876 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:06.491903 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:06.491942 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:06.491973 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:06.518742 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:06.521807 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.521904 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.564550 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:06.667944 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:17:06.668202 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0724 17:17:06.668236 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:06.668327 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:17:06.668787 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0724 17:17:06.668888 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:06.668992 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:06.669148 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:06.669169 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:06.669265 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" },{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" }] }] } I0724 17:17:06.671363 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:06.680281 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:06.680863 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.681096 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.682101 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.683399 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.683530 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:06.683796 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:06.687036 29520 target.go:168] Cloudformation output is in /tmp/test3520948904/out --- PASS: TestContainerdCustom (0.21s) === RUN TestDockerCustom I0724 17:17:06.697924 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:06.698168 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:06.698221 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:06.698311 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:06.698363 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:06.698406 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:06.698434 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:06.698474 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:06.698500 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:06.723719 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:06.726653 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.726739 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.766061 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:06.869023 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0724 17:17:06.869804 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:06.869794 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:06.869866 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:06.869946 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:17:06.869987 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:06.869988 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:06.870014 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:06.870126 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:17:06.870157 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I0724 17:17:06.873307 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:06.881771 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:06.881978 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.882268 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.883594 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.884762 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:06.884995 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:06.885320 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:06.888501 29520 target.go:168] Cloudformation output is in /tmp/test3029026358/out --- PASS: TestDockerCustom (0.20s) === RUN TestAPIServerNodes I0724 17:17:06.898345 29520 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0724 17:17:06.898387 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:06.898603 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:06.898653 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:06.898734 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:06.898784 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:06.898827 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:06.898855 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:06.898892 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:06.898919 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:06.929281 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0724 17:17:06.932268 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.932365 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.932430 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:06.975484 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:07.117350 29520 executor.go:111] Tasks: 0 done / 97 total; 47 can run I0724 17:17:07.117795 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:07.117929 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:07.117949 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:07.118054 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0724 17:17:07.118160 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:07.118335 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:17:07.118356 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:07.118373 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:17:07.118509 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:07.118514 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0724 17:17:07.120897 29520 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0724 17:17:07.130076 29520 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0724 17:17:07.130703 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.130998 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.131294 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.131915 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.132236 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.133858 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.134000 29520 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0724 17:17:07.134334 29520 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0724 17:17:07.137921 29520 target.go:168] Cloudformation output is in /tmp/test3128729529/out I0724 17:17:07.150536 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:07.150766 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:07.150820 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:07.150863 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:07.150919 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:07.150962 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:07.150989 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:07.151025 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:07.151054 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:07.177051 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.10 ********************************************************************************* I0724 17:17:07.179823 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.179906 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.179966 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.220642 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:07.380080 29520 executor.go:111] Tasks: 0 done / 97 total; 47 can run I0724 17:17:07.380774 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:07.381090 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:07.381161 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:07.381813 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:07.382162 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:07.382218 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:07.382322 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:07.386622 29520 executor.go:111] Tasks: 47 done / 97 total; 24 can run I0724 17:17:07.392264 29520 executor.go:111] Tasks: 71 done / 97 total; 23 can run I0724 17:17:07.392489 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.392659 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.392849 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.393678 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.394032 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.395205 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.395322 29520 executor.go:111] Tasks: 94 done / 97 total; 3 can run I0724 17:17:07.395663 29520 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0724 17:17:07.461302 29520 target.go:112] Terraform output is in /tmp/test137612652/out I0724 17:17:07.465784 29520 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.57s) === RUN TestNTHQueueProcessor I0724 17:17:07.465912 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:07.466160 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:07.466215 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:07.466265 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:07.466318 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:07.466410 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:07.466441 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:07.466479 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:07.466509 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:07.491038 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0724 17:17:07.494050 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.494151 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.536993 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:07.661657 29520 executor.go:111] Tasks: 0 done / 99 total; 45 can run W0724 17:17:07.662337 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target W0724 17:17:07.662584 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:07.662648 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:07.662666 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:07.662674 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:07.662767 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:07.668737 29520 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0724 17:17:07.674094 29520 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0724 17:17:07.674533 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.674730 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.675591 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.676964 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.677185 29520 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0724 17:17:07.677550 29520 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0724 17:17:07.677742 29520 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0724 17:17:07.744454 29520 target.go:112] Terraform output is in /tmp/test1129322312/out I0724 17:17:07.749492 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:07.749735 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:07.749795 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:07.749848 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:07.749904 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:07.749947 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:07.749976 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:07.750015 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:07.750046 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:07.774775 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0724 17:17:07.777764 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.777858 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.818709 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0724 17:17:07.944276 29520 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0724 17:17:07.944727 29520 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0724 17:17:07.944859 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:07.944773 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:07.944917 29520 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0724 17:17:07.945158 29520 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0724 17:17:07.945241 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:07.945442 29520 sshkey.go:242] Cloudformation does not manage SSH keys; pre-creating SSH key I0724 17:17:07.945469 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:07.945620 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:07.947515 29520 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0724 17:17:07.952833 29520 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0724 17:17:07.953117 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.953377 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.956950 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.959569 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:07.959731 29520 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0724 17:17:07.960026 29520 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0724 17:17:07.960282 29520 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0724 17:17:07.964044 29520 target.go:168] Cloudformation output is in /tmp/test1725160073/out --- PASS: TestNTHQueueProcessor (0.51s) === RUN TestCustomIRSA I0724 17:17:07.975357 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:07.975608 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:07.975659 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:07.975705 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:07.975753 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:07.975833 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:07.975862 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:07.975902 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:07.975931 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:08.003837 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0724 17:17:08.006824 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.006906 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.045452 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:08.592173 29520 executor.go:111] Tasks: 0 done / 98 total; 50 can run W0724 17:17:08.592671 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:08.592983 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:08.593019 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:08.593144 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0724 17:17:08.593186 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:08.593319 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:08.601622 29520 executor.go:111] Tasks: 50 done / 98 total; 25 can run I0724 17:17:08.606998 29520 executor.go:111] Tasks: 75 done / 98 total; 21 can run I0724 17:17:08.607298 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.607603 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.608431 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.609829 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.609948 29520 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0724 17:17:08.610274 29520 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0724 17:17:08.668480 29520 target.go:112] Terraform output is in /tmp/test843104708/out --- PASS: TestCustomIRSA (0.70s) === RUN TestCustomIRSA119 I0724 17:17:08.680289 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:08.680600 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:08.680662 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:08.680717 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:08.680773 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:08.680861 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:08.680892 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:08.680932 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:08.680963 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:08.709772 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0724 17:17:08.713280 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.713376 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.760216 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:08.860758 29520 executor.go:111] Tasks: 0 done / 96 total; 48 can run I0724 17:17:08.861045 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:08.861072 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:08.861173 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:08.861241 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:08.861333 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0724 17:17:08.861463 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:08.863578 29520 executor.go:111] Tasks: 48 done / 96 total; 25 can run I0724 17:17:08.869048 29520 executor.go:111] Tasks: 73 done / 96 total; 21 can run I0724 17:17:08.869372 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.869722 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.870892 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.871626 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.871745 29520 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0724 17:17:08.872091 29520 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0724 17:17:08.929791 29520 target.go:112] Terraform output is in /tmp/test4082631153/out --- PASS: TestCustomIRSA119 (0.25s) === RUN TestClusterNameDigit I0724 17:17:08.934485 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:08.934740 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:08.934795 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:08.934849 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:08.934902 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:08.934994 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:08.935027 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:08.935067 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:08.935096 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:08.965914 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0724 17:17:08.969546 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:08.969648 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.008196 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:09.112093 29520 executor.go:111] Tasks: 0 done / 93 total; 47 can run I0724 17:17:09.112547 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:09.112610 29520 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0724 17:17:09.112648 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:17:09.112778 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:09.112790 29520 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0724 17:17:09.112880 29520 warmpool.go:124] ASG warm pool is not supported by the terraform target I0724 17:17:09.115556 29520 executor.go:111] Tasks: 47 done / 93 total; 23 can run I0724 17:17:09.120628 29520 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0724 17:17:09.121121 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.121334 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.122197 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.123601 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.123780 29520 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0724 17:17:09.124100 29520 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0724 17:17:09.181042 29520 target.go:112] Terraform output is in /tmp/test677212294/out --- PASS: TestClusterNameDigit (0.25s) === RUN TestLifecycleMinimalAWS I0724 17:17:09.185508 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:09.185769 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:09.185828 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:09.185885 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:09.185940 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:09.185983 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:09.186011 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:09.186050 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:09.186081 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:09.186142 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0724 17:17:09.209960 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:09.212784 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.212882 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:17:09.253872 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:09.351681 29520 iamrole.go:89] ListRoles: { } I0724 17:17:09.353318 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:17:09.353849 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:09.353849 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:09.353932 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:09.354117 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.354280 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.354296 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0724 17:17:09.354433 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0724 17:17:09.354480 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.359208 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.359333 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.359459 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:09.359512 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:09.359550 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:09.359599 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:09.359470 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0724 17:17:09.359653 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:09.359717 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0724 17:17:09.359826 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:09.359877 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.360071 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:09.360246 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:09.360329 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.360444 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:09.361611 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:09.361947 29520 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0724 17:17:09.361929 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:09.362104 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0724 17:17:09.362124 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0724 17:17:09.362267 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0724 17:17:09.362245 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:09.362335 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:09.362339 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0724 17:17:09.362440 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:09.362537 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:09.362568 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0724 17:17:09.362579 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0724 17:17:09.362816 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.363020 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.363144 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.363258 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.363354 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:09.363441 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" }] }], VpcId: "vpc-1" } I0724 17:17:09.363537 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:09.363508 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0724 17:17:09.363595 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:09.363749 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:09.363837 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:09.364041 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:09.364134 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:09.364091 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0724 17:17:09.364218 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:09.364271 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:09.364322 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I0724 17:17:09.364911 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0724 17:17:09.367156 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:09.367431 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" }] }] } I0724 17:17:09.367682 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:09.368075 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:09.368230 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" }] }] } I0724 17:17:09.368664 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I0724 17:17:09.368863 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.369134 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.369334 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:09.369455 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:09.369580 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.369803 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.369922 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.370019 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.370183 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.370363 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0724 17:17:09.370581 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.370811 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:09.370896 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0724 17:17:09.370965 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.371192 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.371438 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:09.371508 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0724 17:17:09.371654 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:09.371792 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.371941 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.374032 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:09.374598 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:09.374669 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:09.374728 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:09.374815 29520 dns.go:238] Pre-creating DNS records I0724 17:17:09.374843 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:09.380296 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:09.383002 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.383097 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0724 17:17:09.421861 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:09.525452 29520 iamrole.go:89] ListRoles: { } I0724 17:17:09.527037 29520 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0724 17:17:09.527516 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:09.527568 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:09.527605 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.527826 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:09.527957 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.529926 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:09.529981 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:09.530043 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I0724 17:17:09.530210 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0724 17:17:09.530388 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:09.534124 29520 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0724 17:17:09.534334 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0724 17:17:09.534348 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.534529 29520 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0724 17:17:09.534515 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:09.534649 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0724 17:17:09.534897 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:09.534963 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.535103 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:09.535199 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] }] } I0724 17:17:09.535341 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:09.535383 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0724 17:17:09.535509 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:09.539058 29520 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0724 17:17:09.539183 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:09.539423 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:09.539633 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.539784 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.539964 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.540193 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:09.540415 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:09.542496 29520 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0724 17:17:09.542738 29520 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0724 17:17:09.545568 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:09.545638 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:09.545675 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:09.545711 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:09.545749 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:09.545805 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:09.545862 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:09.545923 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:09.545981 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:09.546042 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:09.546104 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:09.546183 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:09.546249 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:09.546304 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:09.546372 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:09.546437 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:09.546505 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:09.546566 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:09.546631 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:09.546690 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:09.546750 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:09.546786 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:09.546826 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:09.546863 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:09.546902 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:09.548243 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:09.548275 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:09.548317 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548370 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0724 17:17:09.548427 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548485 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548530 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548586 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548634 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548673 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0724 17:17:09.548724 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548776 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0724 17:17:09.548818 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0724 17:17:09.548866 29520 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0724 17:17:09.548901 29520 api.go:53] PageSize not implemented I0724 17:17:09.548924 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:09.548957 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:09.548977 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:09.549006 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:09.549026 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:09.549109 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:09.549159 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:09.549245 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:09.549314 29520 iamrole.go:89] ListRoles: { } I0724 17:17:09.549419 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:09.549518 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:09.549558 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.minimal.example.com ok I0724 17:17:09.550147 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0724 17:17:09.550215 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0724 17:17:09.550219 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:09.550302 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0724 17:17:09.550374 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0724 17:17:09.550326 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0724 17:17:09.550608 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:09.550648 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } I0724 17:17:09.550388 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } security-group:sg-2 ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0724 17:17:09.550722 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:09.550809 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0724 17:17:09.551041 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:09.551064 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:09.551106 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0724 17:17:09.551174 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0724 17:17:09.551258 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0724 17:17:09.551321 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0724 17:17:09.551437 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:09.551533 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0724 17:17:09.551587 29520 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0724 17:17:09.551603 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0724 17:17:09.551598 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:09.551664 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0724 17:17:09.551704 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0724 17:17:09.551748 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0724 17:17:09.551763 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0724 17:17:09.551792 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0724 17:17:09.551860 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:09.551951 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:09.552037 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:09.556451 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.37s) === RUN TestLifecycleMinimalOpenstack I0724 17:17:09.556733 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0724 17:17:09.566843 29520 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:09.637967 29520 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0724 17:17:09.790088 29520 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0724 17:17:09.790146 29520 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0724 17:17:09.806330 29520 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0724 17:17:09.816820 29520 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0724 17:17:09.892210 29520 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0724 17:17:09.895837 29520 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0724 17:17:09.899355 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:09.899444 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:09.899461 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0724 17:17:09.905059 29520 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:09.952048 29520 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0724 17:17:10.100274 29520 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0724 17:17:10.100349 29520 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0724 17:17:10.119330 29520 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0724 17:17:10.129386 29520 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0724 17:17:10.141504 29520 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0724 17:17:10.143826 29520 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0724 17:17:10.147056 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:10.147193 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:10.147402 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0724 17:17:10.149104 29520 executor.go:111] Tasks: 96 done / 96 total; 0 can run SecurityGroup:22e9354b-ecc0-401f-84ba-7399b094fbee ok SecurityGroup:30f8489e-6b7c-492e-a767-3c8909ff75c7 ok Instance:66d3e039-e191-4d32-955d-1d74b59e28f0 ok Instance:8d1c74e7-e1ea-4cd2-8ca1-1369cef0f444 ok ServerGroup:afb2d527-284e-4a4c-b14d-e13ffc471b07 ok Port:13b407d1-0857-45cb-afc5-2248f1e0055a ok Port:83e6826f-3277-4d68-9c0f-51588f523320 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok ServerGroup:afb7dc1c-6eda-4013-950a-c67ba0d25383 ok Network:a38e8a3b-ae4a-4d62-9ad7-abb2a806029a ok Subnet:6621b19e-6658-4589-b742-411030bb97fe ok SecurityGroup:d72c2c0b-5f72-48a2-9f5a-0d31cdfeeaba ok Instance:c4c760d3-3393-49ab-85fe-6af8ca7eff64 ok Port:a966c154-6dff-4b6d-95fd-434c2afe54be ok --- PASS: TestLifecycleMinimalOpenstack (1.71s) === RUN TestLifecycleMinimalGCE I0724 17:17:11.265168 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0724 17:17:11.267091 29520 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0724 17:17:11.498462 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:17:11.498555 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:17:11.498663 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0724 17:17:11.498698 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0724 17:17:11.498722 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0724 17:17:11.498755 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0724 17:17:11.498829 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0724 17:17:11.500663 29520 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0724 17:17:11.504311 29520 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0724 17:17:11.504384 29520 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0724 17:17:11.513742 29520 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0724 17:17:11.516434 29520 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0724 17:17:11.516667 29520 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0724 17:17:11.516761 29520 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.7 ********************************************************************************* W0724 17:17:11.727475 29520 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0724 17:17:11.727553 29520 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0724 17:17:11.727632 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0724 17:17:11.727661 29520 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0724 17:17:11.727681 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0724 17:17:11.727707 29520 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0724 17:17:11.727766 29520 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0724 17:17:11.729159 29520 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0724 17:17:11.737203 29520 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0724 17:17:11.742875 29520 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0724 17:17:11.746468 29520 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0724 17:17:11.746645 29520 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0724 17:17:11.748298 29520 gce.go:98] Scanning zones: [us-test1-a] I0724 17:17:11.748666 29520 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0724 17:17:11.748690 29520 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1658683031 ok InstanceTemplate:nodes-minimal-gce-example-com-1658683031 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.49s) === RUN TestLifecycleFloatingIPOpenstack I0724 17:17:11.750478 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0724 17:17:11.758821 29520 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:11.825058 29520 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0724 17:17:11.970442 29520 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0724 17:17:11.970494 29520 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0724 17:17:11.986323 29520 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0724 17:17:11.995489 29520 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0724 17:17:12.046301 29520 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0724 17:17:12.050666 29520 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0724 17:17:12.052998 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:12.053307 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:12.053363 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0724 17:17:12.057767 29520 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:12.105254 29520 apply_cluster.go:467] Gossip DNS: skipping DNS validation W0724 17:17:12.251969 29520 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0724 17:17:12.252023 29520 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0724 17:17:12.276866 29520 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0724 17:17:12.280536 29520 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0724 17:17:12.292653 29520 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0724 17:17:12.295242 29520 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0724 17:17:12.298570 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:12.298864 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. W0724 17:17:12.298939 29520 bootstrapscript.go:166] exporting username and password. Consider using application credentials instead. I0724 17:17:12.300860 29520 executor.go:111] Tasks: 98 done / 98 total; 0 can run Instance:7041f682-bb3c-4ade-b550-6a9946a19fa2 ok Router-IF:dfa3b595-18c2-4aef-951a-b526b4aeaa78 ok Subnet:dfa3b595-18c2-4aef-951a-b526b4aeaa78 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Router:6fe3a0ce-7ec4-4e74-ac70-45706ece36d9 ok Port:3187a33e-3d34-47a5-a64f-61e0fc24f888 ok SecurityGroup:741fcfc0-51e5-42f7-bced-ec59a77edd79 ok Port:52f8cb80-446f-4d63-a067-8d52c54e9bfb ok Instance:d1677095-487b-433b-ad07-3974fa3fa614 ok ServerGroup:75a4455e-2586-4278-a3b4-d293d87c0fbf ok Network:17fba162-ddf1-491c-971a-df1745d24b12 ok Port:cb1ff02b-fa2e-4f66-bd91-e94475db89ea ok Port:e1364a9e-b08d-4b64-89ba-6f821e551a16 ok Instance:06e247db-939a-4296-b78f-e1a87e20ff1d ok ServerGroup:51728a3d-0694-4201-acce-2ed2226ab544 ok SecurityGroup:474e0fee-cf3f-4e10-b179-37ff6f544c85 ok SecurityGroup:4345a126-ac53-4377-985b-2347516398b1 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.67s) === RUN TestLifecyclePrivateCalico I0724 17:17:13.416828 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:13.417129 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:13.417182 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:13.417227 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:13.417273 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:13.417315 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:13.417343 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:13.417382 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:13.417410 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:13.417465 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0724 17:17:13.453122 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:17:13.456516 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.456622 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.456685 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.500127 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:13.903412 29520 iamrole.go:89] ListRoles: { } I0724 17:17:13.905336 29520 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0724 17:17:13.905529 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.905703 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.905799 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:13.905789 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0724 17:17:13.905875 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0724 17:17:13.905950 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0724 17:17:13.906006 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0724 17:17:13.906206 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:13.907393 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.907710 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:13.907916 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:13.907945 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0724 17:17:13.908121 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0724 17:17:13.909396 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.909692 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.909852 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:13.910135 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:13.910158 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.910418 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:13.912372 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:13.912456 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.912782 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:13.912855 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:13.912916 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:13.913000 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:13.913750 29520 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0724 17:17:13.913898 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:13.913971 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0724 17:17:13.914014 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0724 17:17:13.914011 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.914147 29520 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0724 17:17:13.914211 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0724 17:17:13.914144 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0724 17:17:13.914258 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:13.914271 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.914369 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0724 17:17:13.914470 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.914485 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0724 17:17:13.914576 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:13.914629 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0724 17:17:13.914816 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:13.914917 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:13.914891 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0724 17:17:13.914966 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:13.915009 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.915198 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:13.915186 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0724 17:17:13.915395 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:13.915504 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:13.915364 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0724 17:17:13.915536 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0724 17:17:13.915647 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.915755 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:13.915856 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:13.915932 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0724 17:17:13.916047 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0724 17:17:13.916150 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:13.916222 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:13.916265 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:13.916320 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:13.916360 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:13.916496 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0724 17:17:13.916502 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:13.916617 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:13.916669 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:13.916707 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0724 17:17:13.916355 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0724 17:17:13.916828 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0724 17:17:13.916869 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:13.916961 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:13.917072 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:13.917146 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:13.917213 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:13.917339 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:13.917399 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:13.917459 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:13.917509 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0724 17:17:13.917596 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:13.917663 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:13.917721 29520 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0724 17:17:13.919637 29520 executor.go:111] Tasks: 78 done / 124 total; 40 can run W0724 17:17:13.919791 29520 api.go:53] PageSize not implemented I0724 17:17:13.919867 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0724 17:17:13.919938 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0724 17:17:13.920063 29520 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0724 17:17:13.920133 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0724 17:17:13.920037 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.920168 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0724 17:17:13.920259 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0724 17:17:13.920225 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0724 17:17:13.920405 29520 api.go:53] PageSize not implemented I0724 17:17:13.920426 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0724 17:17:13.920443 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:13.920599 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0724 17:17:13.920636 29520 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0724 17:17:13.920692 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0724 17:17:13.920723 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0724 17:17:13.920782 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0724 17:17:13.920709 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.920891 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.921141 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.921334 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.921487 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.921677 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.921858 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:13.921938 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:13.921998 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:13.922049 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:13.922110 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.922232 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:13.922303 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.922414 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.922453 29520 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.922522 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0724 17:17:13.922679 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:13.922811 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.922955 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0724 17:17:13.923053 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" }] }] } I0724 17:17:13.923189 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.923297 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.923366 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.923557 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:13.923736 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0724 17:17:13.923870 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" }] }] } I0724 17:17:13.924054 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.924220 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:13.924300 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:13.924352 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0724 17:17:13.924424 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.924620 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0724 17:17:13.924680 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0724 17:17:13.924728 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:13.924909 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.925012 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.925088 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:13.925170 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.925342 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.925515 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0724 17:17:13.925669 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.925864 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0724 17:17:13.927457 29520 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0724 17:17:13.927725 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:13.927812 29520 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:13.927720 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:13.927947 29520 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0724 17:17:13.928043 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0724 17:17:13.928089 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:13.927990 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:13.928133 29520 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0724 17:17:13.928189 29520 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0724 17:17:13.928258 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:13.928403 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0724 17:17:13.928474 29520 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0724 17:17:13.928513 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:13.928544 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:13.928591 29520 dns.go:238] Pre-creating DNS records I0724 17:17:13.928617 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:13.934918 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* I0724 17:17:13.938204 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.938300 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.938366 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:13.977435 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:14.397047 29520 iamrole.go:89] ListRoles: { } I0724 17:17:14.399366 29520 executor.go:111] Tasks: 0 done / 124 total; 50 can run I0724 17:17:14.399588 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.399769 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.399958 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.400137 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.400333 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:14.400395 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:14.400341 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:14.400478 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.400529 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:14.400642 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:14.402102 29520 executor.go:111] Tasks: 50 done / 124 total; 28 can run I0724 17:17:14.402388 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0724 17:17:14.402306 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0724 17:17:14.402538 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0724 17:17:14.402466 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.402617 29520 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0724 17:17:14.402652 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.402816 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:14.402863 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0724 17:17:14.403030 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:14.403100 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.403359 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:14.403433 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:14.403526 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0724 17:17:14.403797 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:14.403880 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.404016 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0724 17:17:14.404200 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:14.404259 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.404352 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0724 17:17:14.404500 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:14.404573 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.404708 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.408217 29520 executor.go:111] Tasks: 78 done / 124 total; 40 can run W0724 17:17:14.408560 29520 api.go:53] PageSize not implemented I0724 17:17:14.408527 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.408607 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0724 17:17:14.408780 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0724 17:17:14.408752 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.408848 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0724 17:17:14.408923 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0724 17:17:14.409037 29520 api.go:53] PageSize not implemented I0724 17:17:14.409062 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0724 17:17:14.409050 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.409152 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0724 17:17:14.409187 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0724 17:17:14.409175 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0724 17:17:14.409291 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0724 17:17:14.409333 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0724 17:17:14.409426 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.409563 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.409628 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.409732 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.409833 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.409913 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.410055 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.412528 29520 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0724 17:17:14.412625 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:14.412708 29520 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0724 17:17:14.412737 29520 api.go:53] PageSize not implemented I0724 17:17:14.412662 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.412772 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0724 17:17:14.412818 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0724 17:17:14.412954 29520 api.go:53] PageSize not implemented I0724 17:17:14.412978 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0724 17:17:14.413062 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0724 17:17:14.413152 29520 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0724 17:17:14.413233 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.413339 29520 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0724 17:17:14.416993 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:14.417064 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:14.417110 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:14.417165 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:14.417215 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0724 17:17:14.417252 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:14.417285 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:14.417317 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:14.417351 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:14.417387 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:14.417425 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:14.417463 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:14.417503 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0724 17:17:14.417543 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0724 17:17:14.417578 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:14.417612 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:14.417647 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:14.417683 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:14.417719 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:14.417758 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:14.417794 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:14.417831 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:14.417867 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:14.417903 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:14.417939 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:14.417976 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:14.418009 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:14.418047 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:14.418080 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:14.418112 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:14.418150 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:14.418186 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:14.418221 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:14.419882 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:14.419909 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:14.419945 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420008 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0724 17:17:14.420087 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420145 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420194 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420245 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420286 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420372 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0724 17:17:14.420437 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420500 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:14.420529 29520 natgateway.go:117] DescribeNatGateways: { } I0724 17:17:14.420565 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0724 17:17:14.420606 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0724 17:17:14.420666 29520 api.go:53] PageSize not implemented I0724 17:17:14.420685 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0724 17:17:14.420739 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:14.420767 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:14.420784 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:14.420808 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:14.420827 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:14.420884 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:14.420910 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:14.420951 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:14.420992 29520 iamrole.go:89] ListRoles: { } I0724 17:17:14.421035 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:14.421083 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:14.421123 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0724 17:17:14.421162 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0724 17:17:14.421195 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0724 17:17:14.424225 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-group:bastion.privatecalico.example.com ok I0724 17:17:14.424347 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0724 17:17:14.424500 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-group:nodes.privatecalico.example.com ok I0724 17:17:14.424555 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0724 17:17:14.424647 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0724 17:17:14.424631 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } autoscaling-config:lt-2 ok I0724 17:17:14.424843 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0724 17:17:14.424902 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } load-balancer:bastion-privatecalico-exa-hocohm ok iam-instance-profile:nodes.privatecalico.example.com ok I0724 17:17:14.424893 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:14.425003 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0724 17:17:14.425023 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0724 17:17:14.425335 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0724 17:17:14.425677 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0724 17:17:14.425748 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0724 17:17:14.425030 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0724 17:17:14.425946 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.425965 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } I0724 17:17:14.426013 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:14.426072 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0724 17:17:14.426194 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.426207 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } iam-instance-profile:bastions.privatecalico.example.com ok security-group:sg-3 ok autoscaling-config:lt-3 ok security-group:sg-1 ok I0724 17:17:14.426896 29520 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-config:lt-1 ok nat-gateway:nat-1 ok I0724 17:17:14.426998 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:14.427083 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:14.425067 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0724 17:17:14.427139 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0724 17:17:14.427195 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0724 17:17:14.427317 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0724 17:17:14.427367 29520 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0724 17:17:14.427383 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0724 17:17:14.427335 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } I0724 17:17:14.427415 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } subnet:subnet-2 ok iam-role:masters.privatecalico.example.com ok I0724 17:17:14.427434 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:14.427476 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0724 17:17:14.427567 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0724 17:17:14.427597 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0724 17:17:14.427659 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0724 17:17:14.427539 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } W0724 17:17:14.427736 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.427716 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0724 17:17:14.427766 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:14.427794 29520 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0724 17:17:14.427815 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0724 17:17:14.427865 29520 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } I0724 17:17:14.427831 29520 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } iam-role:bastions.privatecalico.example.com ok elastic-ip:eipalloc-1 ok I0724 17:17:14.427936 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:14.428033 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 } ] } W0724 17:17:14.428200 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.428223 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0724 17:17:14.428270 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:14.428375 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:14.428438 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0724 17:17:14.428516 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:14.428595 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:14.428872 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.01s) === RUN TestLifecyclePrivateKopeio I0724 17:17:14.429160 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:14.429427 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:14.429471 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:14.429513 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:14.429560 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:14.429599 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:14.429628 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:14.429664 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:14.429690 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:14.429744 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0724 17:17:14.459865 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:14.462937 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.463031 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.463095 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.503404 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:14.620330 29520 iamrole.go:89] ListRoles: { } I0724 17:17:14.622640 29520 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0724 17:17:14.623033 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.623136 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0724 17:17:14.623367 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:14.623495 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0724 17:17:14.623439 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:14.623728 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:14.623823 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.623922 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0724 17:17:14.624061 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0724 17:17:14.624077 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.624310 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.624424 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0724 17:17:14.624436 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.624577 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:14.624560 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0724 17:17:14.624645 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:14.624672 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.624726 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:14.624975 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:14.625046 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:14.625126 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:14.625165 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:14.625195 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:14.625237 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.625285 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.626048 29520 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0724 17:17:14.626242 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0724 17:17:14.626212 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.626371 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.626455 29520 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0724 17:17:14.626494 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.626608 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0724 17:17:14.626642 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:14.626682 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0724 17:17:14.626710 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:14.626724 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0724 17:17:14.626949 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0724 17:17:14.627056 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0724 17:17:14.627043 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:14.627214 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:14.627133 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0724 17:17:14.627305 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:14.627378 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0724 17:17:14.627610 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0724 17:17:14.627586 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0724 17:17:14.627874 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:14.627989 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.628096 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.628188 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.628273 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.628360 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.628261 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0724 17:17:14.628343 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:14.628448 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0724 17:17:14.628551 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0724 17:17:14.628657 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.628734 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.628823 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.628927 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:14.629018 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:14.628642 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0724 17:17:14.629082 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:14.629033 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:14.629173 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0724 17:17:14.629359 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0724 17:17:14.629352 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0724 17:17:14.629557 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:14.629546 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0724 17:17:14.629673 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:14.629768 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:14.629831 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:14.629888 29520 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0724 17:17:14.630002 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:14.630111 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0724 17:17:14.630199 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:14.630246 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:14.630328 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:14.630394 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:14.630428 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0724 17:17:14.630451 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0724 17:17:14.630586 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:14.630637 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0724 17:17:14.630687 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:14.630729 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:14.630781 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:14.630824 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:14.630874 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:14.630910 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:14.630950 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0724 17:17:14.630997 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0724 17:17:14.631030 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:14.631078 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0724 17:17:14.631122 29520 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] } I0724 17:17:14.631575 29520 executor.go:111] Tasks: 78 done / 123 total; 39 can run W0724 17:17:14.631690 29520 api.go:53] PageSize not implemented I0724 17:17:14.631787 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.631841 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } W0724 17:17:14.631949 29520 api.go:53] PageSize not implemented I0724 17:17:14.631910 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0724 17:17:14.632000 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0724 17:17:14.632099 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0724 17:17:14.632200 29520 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] } I0724 17:17:14.632303 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0724 17:17:14.632347 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0724 17:17:14.632404 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0724 17:17:14.632356 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.632501 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.632652 29520 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0724 17:17:14.632719 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.632651 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:14.632754 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0724 17:17:14.632799 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0724 17:17:14.632885 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0724 17:17:14.632996 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" }] }] } I0724 17:17:14.633153 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.633247 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.633315 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.633389 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.633463 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0724 17:17:14.633509 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:14.634223 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:14.634626 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0724 17:17:14.634888 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:14.635090 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.635239 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.635318 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:14.635772 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.636087 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:14.636185 29520 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0724 17:17:14.636270 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:14.636386 29520 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0724 17:17:14.636476 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:14.636570 29520 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0724 17:17:14.636663 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.636821 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0724 17:17:14.637263 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.639548 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.639714 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.639869 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0724 17:17:14.639915 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.640000 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0724 17:17:14.640134 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.640259 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.640400 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0724 17:17:14.640439 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.640581 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0724 17:17:14.640627 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:14.640757 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:14.640798 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I0724 17:17:14.640938 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.641074 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0724 17:17:14.641206 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0724 17:17:14.641298 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0724 17:17:14.641418 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.641493 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:14.641565 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.641611 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0724 17:17:14.641649 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.641693 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0724 17:17:14.641730 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0724 17:17:14.641776 29520 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0724 17:17:14.641828 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0724 17:17:14.641875 29520 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0724 17:17:14.641916 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:14.641957 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0724 17:17:14.642018 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0724 17:17:14.642056 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0724 17:17:14.643879 29520 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0724 17:17:14.644113 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.644128 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:14.644287 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0724 17:17:14.644301 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:14.644346 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.644466 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0724 17:17:14.644533 29520 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0724 17:17:14.644573 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:14.644606 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:14.644655 29520 dns.go:238] Pre-creating DNS records I0724 17:17:14.644681 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:14.651523 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:14.655099 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.655193 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.655256 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.696427 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:14.803328 29520 iamrole.go:89] ListRoles: { } I0724 17:17:14.808919 29520 executor.go:111] Tasks: 0 done / 123 total; 47 can run I0724 17:17:14.809135 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] }] } I0724 17:17:14.809417 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:14.809504 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.809693 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:14.809750 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:14.809742 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:14.809825 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.809922 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:14.810134 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.810342 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.812003 29520 executor.go:111] Tasks: 47 done / 123 total; 31 can run I0724 17:17:14.812199 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.812451 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0724 17:17:14.812501 29520 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0724 17:17:14.812414 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0724 17:17:14.812566 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0724 17:17:14.812588 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.812786 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0724 17:17:14.812834 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.812965 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:14.813004 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:14.813060 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0724 17:17:14.813217 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:14.813274 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.813414 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:14.813466 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.813616 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.813781 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:14.813825 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.813950 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.814079 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0724 17:17:14.814245 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:14.814296 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.814476 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0724 17:17:14.814635 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:14.814706 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:14.814804 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0724 17:17:14.814949 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:14.817903 29520 executor.go:111] Tasks: 78 done / 123 total; 39 can run I0724 17:17:14.818218 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0724 17:17:14.818359 29520 api.go:53] PageSize not implemented I0724 17:17:14.818344 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.818406 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.818606 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0724 17:17:14.818560 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.818702 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0724 17:17:14.818782 29520 api.go:53] PageSize not implemented I0724 17:17:14.818768 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.818831 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.819020 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.818983 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.819104 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0724 17:17:14.819266 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.819326 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0724 17:17:14.819420 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0724 17:17:14.819514 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.819590 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.819652 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.819722 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0724 17:17:14.819799 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.819934 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:14.821976 29520 executor.go:111] Tasks: 117 done / 123 total; 6 can run I0724 17:17:14.822155 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:14.822298 29520 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" W0724 17:17:14.822233 29520 api.go:53] PageSize not implemented I0724 17:17:14.822252 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:14.822455 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:14.822406 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } W0724 17:17:14.822682 29520 api.go:53] PageSize not implemented I0724 17:17:14.822711 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.822824 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.822929 29520 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0724 17:17:14.826744 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:14.826811 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:14.826852 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:14.826888 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:14.826927 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:14.826962 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:14.827022 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:14.827064 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:14.827103 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:14.827141 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:14.827181 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:14.827221 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0724 17:17:14.827267 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:14.827304 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:14.827340 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:14.827379 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:14.827414 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:14.827452 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0724 17:17:14.827490 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:14.827528 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:14.827565 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:14.827601 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:14.827640 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:14.827678 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:14.827716 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:14.827750 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:14.827790 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0724 17:17:14.827828 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0724 17:17:14.827867 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:14.827902 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:14.827937 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:14.827977 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:14.828015 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:14.828052 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:14.829779 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:14.829811 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:14.829854 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.829923 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0724 17:17:14.830006 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830070 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830117 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830171 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830215 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830267 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0724 17:17:14.830366 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830442 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:14.830478 29520 natgateway.go:117] DescribeNatGateways: { } I0724 17:17:14.830517 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0724 17:17:14.830561 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0724 17:17:14.830616 29520 api.go:53] PageSize not implemented I0724 17:17:14.830633 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0724 17:17:14.830687 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:14.830714 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:14.830734 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:14.830760 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:14.830778 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:14.830831 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:14.830858 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:14.830908 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:14.830948 29520 iamrole.go:89] ListRoles: { } I0724 17:17:14.830995 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:14.831046 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:14.831096 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-1","rtb-2"] } I0724 17:17:14.831140 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0724 17:17:14.831172 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0724 17:17:14.831819 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0724 17:17:14.831890 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:14.831928 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0724 17:17:14.831984 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } autoscaling-group:nodes.privatekopeio.example.com ok iam-instance-profile:masters.privatekopeio.example.com ok autoscaling-group:bastion.privatekopeio.example.com ok I0724 17:17:14.832130 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0724 17:17:14.832008 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatekopeio.example.com. ok I0724 17:17:14.832169 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0724 17:17:14.832245 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0724 17:17:14.832279 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0724 17:17:14.831993 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0724 17:17:14.832369 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.832334 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0724 17:17:14.832436 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0724 17:17:14.832397 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-3 ok I0724 17:17:14.832574 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0724 17:17:14.832668 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0724 17:17:14.832799 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:14.832899 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0724 17:17:14.833008 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.833032 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0724 17:17:14.833080 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:14.833138 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:14.833195 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0724 17:17:14.833310 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.833326 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:14.833385 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:14.833500 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0724 17:17:14.833561 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:14.833529 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0724 17:17:14.833696 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0724 17:17:14.833714 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0724 17:17:14.833627 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0724 17:17:14.833771 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.833758 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0724 17:17:14.833795 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } I0724 17:17:14.833814 29520 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com security-group:sg-4 ok I0724 17:17:14.833834 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0724 17:17:14.833869 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0724 17:17:14.833851 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0724 17:17:14.833895 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0724 17:17:14.833940 29520 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0724 17:17:14.833919 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:14.833953 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0724 17:17:14.833983 29520 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0724 17:17:14.834017 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0724 17:17:14.833968 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0724 17:17:14.834051 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:14.834072 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0724 17:17:14.834137 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:14.834190 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0724 17:17:14.834281 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0724 17:17:14.834343 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:14.834399 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0724 17:17:14.834473 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:14.834552 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:14.834834 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.41s) === RUN TestLifecycleIPv6 I0724 17:17:14.835144 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:14.835404 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:14.835449 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:14.835495 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:14.835542 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:14.835582 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:14.835611 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:14.835672 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:14.835698 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:14.835751 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0724 17:17:14.861721 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:14.867323 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.867431 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:14.907764 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.076072 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.077775 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0724 17:17:15.078301 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0724 17:17:15.078394 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0724 17:17:15.078321 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.078721 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:15.078653 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.080331 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.080481 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0724 17:17:15.080625 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" }] } I0724 17:17:15.080571 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.080779 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:15.081555 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.081757 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:15.081838 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.081908 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:15.081976 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.082181 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:15.082318 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:15.082386 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:15.082445 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:15.082541 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.082608 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.082799 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:15.083131 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.085803 29520 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0724 17:17:15.085944 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0724 17:17:15.086024 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] }] } I0724 17:17:15.086048 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0724 17:17:15.086155 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0724 17:17:15.086270 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0724 17:17:15.086278 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0724 17:17:15.086372 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0724 17:17:15.086488 29520 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0724 17:17:15.086626 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:15.086702 29520 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0724 17:17:15.086729 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:15.086773 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0724 17:17:15.087007 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0724 17:17:15.087125 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:15.087184 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.087230 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0724 17:17:15.087190 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0724 17:17:15.087369 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0724 17:17:15.087388 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.087568 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:15.087642 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.087709 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0724 17:17:15.087812 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:15.087857 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.087933 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.088000 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:15.088088 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.088135 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:15.088202 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.088301 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:15.088355 29520 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0724 17:17:15.088448 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:15.088501 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:15.088556 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:15.088601 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:15.088683 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] } I0724 17:17:15.088593 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0724 17:17:15.091410 29520 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0724 17:17:15.091657 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.091854 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.092077 29520 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }] }] } I0724 17:17:15.092159 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.092381 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:15.092518 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.092571 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:15.092699 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.092874 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0724 17:17:15.092948 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.093018 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.093082 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.093157 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.093325 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.093468 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.093606 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.093773 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.093920 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.094049 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.094175 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.094266 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0724 17:17:15.094365 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.094494 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.094615 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:15.094659 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.094787 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" }] }] } I0724 17:17:15.094916 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0724 17:17:15.094959 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:15.095090 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:15.095132 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.095262 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0724 17:17:15.095348 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0724 17:17:15.095433 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.095511 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0724 17:17:15.095638 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0724 17:17:15.095764 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" }] }] } I0724 17:17:15.095891 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.095938 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.098602 29520 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0724 17:17:15.098738 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:15.098812 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:15.098791 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.098969 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:15.098932 29520 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }], Type: "network" } I0724 17:17:15.099019 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0724 17:17:15.099047 29520 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0724 17:17:15.099060 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.099132 29520 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.099129 29520 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0724 17:17:15.099256 29520 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0724 17:17:15.099350 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0724 17:17:15.099394 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.099445 29520 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0724 17:17:15.099512 29520 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0724 17:17:15.099583 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.099605 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.099706 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0724 17:17:15.099696 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.099882 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.099990 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.100085 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:17:15.100128 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.100161 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.100215 29520 dns.go:238] Pre-creating DNS records I0724 17:17:15.100240 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.106082 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:15.109418 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.109515 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.148572 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.316218 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.318113 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0724 17:17:15.318479 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:15.318481 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.318697 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.318875 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I0724 17:17:15.319175 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:15.319253 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:15.319344 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.319409 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.319514 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.319737 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.320836 29520 executor.go:111] Tasks: 45 done / 104 total; 21 can run I0724 17:17:15.320977 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.321128 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.321181 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0724 17:17:15.321251 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.321284 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0724 17:17:15.321253 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0724 17:17:15.321336 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0724 17:17:15.321444 29520 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0724 17:17:15.321537 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.321598 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.321722 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0724 17:17:15.321892 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:15.321944 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0724 17:17:15.322153 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.322214 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.326042 29520 executor.go:111] Tasks: 66 done / 104 total; 30 can run I0724 17:17:15.326197 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.326295 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.326492 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.326642 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.326797 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.326946 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:15.327302 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.327400 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0724 17:17:15.327441 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0724 17:17:15.327485 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.327623 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.327893 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.328071 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.328376 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0724 17:17:15.330512 29520 executor.go:111] Tasks: 96 done / 104 total; 5 can run I0724 17:17:15.330604 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.330645 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:15.330729 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:15.330701 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.330749 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0724 17:17:15.330770 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0724 17:17:15.330795 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0724 17:17:15.330829 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0724 17:17:15.330875 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0724 17:17:15.330910 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0724 17:17:15.331018 29520 executor.go:111] Tasks: 101 done / 104 total; 3 can run I0724 17:17:15.331103 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.331097 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.331213 29520 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0724 17:17:15.331239 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:15.331263 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0724 17:17:15.331266 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.331361 29520 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0724 17:17:15.331385 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:15.331411 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0724 17:17:15.331513 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:17:15.334699 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:15.334760 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:15.334798 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:15.334831 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:15.334868 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0724 17:17:15.334902 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:15.334935 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:15.334967 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:15.335000 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:15.335036 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:15.335075 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:15.335115 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:15.335156 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0724 17:17:15.335192 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:15.335224 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:15.335255 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:15.335292 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:15.335325 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:15.335360 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:15.335394 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:15.335428 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:15.335463 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.335501 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:15.335533 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:15.335565 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:15.335596 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:15.335636 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:15.335672 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:15.335706 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:15.337334 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:15.337363 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:15.337403 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337457 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0724 17:17:15.337517 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337574 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337623 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337675 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337715 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337757 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0724 17:17:15.337809 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337856 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:15.337882 29520 natgateway.go:117] DescribeNatGateways: { } I0724 17:17:15.337922 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0724 17:17:15.337964 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0724 17:17:15.338017 29520 api.go:53] PageSize not implemented I0724 17:17:15.338035 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:15.338060 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:15.338074 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0724 17:17:15.338115 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:15.338137 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:15.338153 29520 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0724 17:17:15.338212 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.338239 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.338282 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:15.338316 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.338358 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:15.338407 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:15.338449 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0724 17:17:15.338486 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0724 17:17:15.338520 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-config:lt-1 ok I0724 17:17:15.339125 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.339141 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0724 17:17:15.339219 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok autoscaling-group:nodes.minimal-ipv6.example.com ok autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0724 17:17:15.339233 29520 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0724 17:17:15.339315 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0724 17:17:15.339388 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0724 17:17:15.339439 29520 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0724 17:17:15.339195 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0724 17:17:15.339525 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:15.339542 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:15.339356 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I0724 17:17:15.339577 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:15.339633 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0724 17:17:15.339781 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:15.339803 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok I0724 17:17:15.339873 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0724 17:17:15.339935 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0724 17:17:15.339991 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:15.340038 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.340090 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:15.340123 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:15.340191 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0724 17:17:15.340300 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0724 17:17:15.340381 29520 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0724 17:17:15.340498 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:15.340553 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0724 17:17:15.340634 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0724 17:17:15.340573 29520 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0724 17:17:15.340660 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0724 17:17:15.340748 29520 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0724 17:17:15.340764 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0724 17:17:15.340798 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0724 17:17:15.340838 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0724 17:17:15.340867 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0724 17:17:15.340950 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:15.341036 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:15.341128 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:15.341391 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.51s) === RUN TestLifecycleSharedVPC I0724 17:17:15.341676 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:15.341927 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:15.341972 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:15.342009 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:15.342053 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:15.342094 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:15.342123 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:15.342157 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:15.342185 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:15.342238 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0724 17:17:15.370814 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:15.374091 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.374205 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.413490 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.510647 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.512278 29520 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0724 17:17:15.512586 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0724 17:17:15.512738 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I0724 17:17:15.512965 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:15.513967 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:15.513097 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0724 17:17:15.514499 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0724 17:17:15.514510 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.514754 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.514930 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.515021 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:15.515063 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:15.515121 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.515236 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:15.515482 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:15.515602 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:15.515991 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:15.516072 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.521271 29520 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0724 17:17:15.521525 29520 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0724 17:17:15.521670 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0724 17:17:15.521817 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0724 17:17:15.521691 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.521881 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0724 17:17:15.521962 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.521971 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0724 17:17:15.522093 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:15.522228 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0724 17:17:15.522189 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0724 17:17:15.522486 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.522388 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0724 17:17:15.522555 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:15.522728 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:15.522868 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:15.522938 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.522996 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0724 17:17:15.523254 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:15.523374 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:15.523432 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.523482 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:15.523527 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] } I0724 17:17:15.523605 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:15.524083 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0724 17:17:15.526866 29520 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0724 17:17:15.527059 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.527226 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0724 17:17:15.527421 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:15.527473 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0724 17:17:15.527591 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.527737 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.527868 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0724 17:17:15.527936 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0724 17:17:15.528091 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.528219 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0724 17:17:15.528286 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.528425 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.528569 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0724 17:17:15.528703 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.528857 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:15.528985 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:15.529110 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.529245 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.529353 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.529421 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.529628 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.529777 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0724 17:17:15.529915 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:15.530061 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.530166 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.531660 29520 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0724 17:17:15.531920 29520 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0724 17:17:15.531969 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.532016 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.532068 29520 dns.go:238] Pre-creating DNS records I0724 17:17:15.532095 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.537745 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:15.540527 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.540623 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.579476 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.677405 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.678887 29520 executor.go:111] Tasks: 0 done / 82 total; 42 can run I0724 17:17:15.679426 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:15.679364 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.679619 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.679782 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.679934 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:15.680186 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:15.680362 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:15.680402 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:15.682366 29520 executor.go:111] Tasks: 42 done / 82 total; 17 can run I0724 17:17:15.682581 29520 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0724 17:17:15.682507 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:15.682771 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0724 17:17:15.682706 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.682862 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0724 17:17:15.683022 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.683108 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.683324 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:15.683389 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0724 17:17:15.683588 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.691613 29520 executor.go:111] Tasks: 59 done / 82 total; 21 can run I0724 17:17:15.691849 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.691981 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.692144 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.692347 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.692564 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.692741 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.692865 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.694528 29520 executor.go:111] Tasks: 80 done / 82 total; 2 can run I0724 17:17:15.694758 29520 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0724 17:17:15.697503 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:15.697566 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:15.697602 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:15.697633 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:15.697668 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:15.697700 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:15.697732 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:15.697768 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:15.697808 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:15.697847 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:15.697880 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:15.697910 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:15.697943 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:15.697973 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:15.698007 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:15.698042 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.698078 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:15.698110 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:15.698140 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:15.698172 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:15.698205 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:15.698240 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:15.700063 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:15.700093 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:15.700134 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700186 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0724 17:17:15.700241 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700300 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700364 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700406 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700446 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700484 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0724 17:17:15.700530 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700587 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0724 17:17:15.700623 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0724 17:17:15.700669 29520 api.go:53] PageSize not implemented I0724 17:17:15.700689 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:15.700715 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:15.700734 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:15.700758 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:15.700775 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:15.700825 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.700851 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.700894 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:15.700932 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.700972 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:15.701010 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:15.701037 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok autoscaling-config:lt-1 ok I0724 17:17:15.701508 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-group:nodes.sharedvpc.example.com ok I0724 17:17:15.701606 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.701596 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0724 17:17:15.701503 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedvpc.example.com. ok I0724 17:17:15.701708 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0724 17:17:15.701674 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0724 17:17:15.701893 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:15.701849 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0724 17:17:15.701932 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } iam-instance-profile:nodes.sharedvpc.example.com ok security-group:sg-2 ok I0724 17:17:15.702038 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0724 17:17:15.702081 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:15.702240 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0724 17:17:15.702340 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.702440 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0724 17:17:15.702750 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:15.702771 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0724 17:17:15.702872 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0724 17:17:15.702925 29520 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0724 17:17:15.702941 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0724 17:17:15.702909 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:15.702981 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0724 17:17:15.703011 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0724 17:17:15.703071 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0724 17:17:15.703086 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0724 17:17:15.703121 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0724 17:17:15.703199 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:15.703488 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.36s) === RUN TestLifecycleComplex I0724 17:17:15.703857 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:15.704165 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:15.704212 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:15.704253 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:15.704301 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:15.704343 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:15.704370 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:15.704405 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:15.704432 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:15.704488 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0724 17:17:15.739140 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:15.743015 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.743116 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.785099 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.889547 29520 iamrole.go:89] ListRoles: { } I0724 17:17:15.891818 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0724 17:17:15.892153 29520 api.go:53] PageSize not implemented I0724 17:17:15.892258 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:15.892414 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.892503 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0724 17:17:15.892610 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0724 17:17:15.892662 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:15.892797 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:15.892859 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.892932 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0724 17:17:15.893090 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" } ] } I0724 17:17:15.893207 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:15.893412 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:15.893584 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:15.893723 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:15.893898 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.894153 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.894418 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:15.894464 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0724 17:17:15.894604 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:15.894640 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:15.894673 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:15.894730 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:15.894834 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.894990 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:15.895042 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.895359 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:17:15.895506 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:15.895542 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.895664 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:15.895774 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0724 17:17:15.895772 29520 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0724 17:17:15.895835 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:15.895967 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:15.896092 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:15.896118 29520 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0724 17:17:15.896099 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0724 17:17:15.896197 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0724 17:17:15.896306 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:15.896355 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.896414 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0724 17:17:15.896596 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0724 17:17:15.896713 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.896781 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0724 17:17:15.896862 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:15.896814 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0724 17:17:15.897025 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:15.897070 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0724 17:17:15.897110 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] } I0724 17:17:15.897304 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:15.897388 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:15.897468 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:15.897538 29520 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0724 17:17:15.897698 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:15.897884 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.898036 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.898138 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0724 17:17:15.898180 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.898304 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:15.898382 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.898355 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0724 17:17:15.898498 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:15.898623 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:15.898783 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0724 17:17:15.898958 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0724 17:17:15.899116 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0724 17:17:15.899297 29520 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0724 17:17:15.899381 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0724 17:17:15.899459 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:15.899524 29520 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0724 17:17:15.899591 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:15.899655 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0724 17:17:15.899225 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0724 17:17:15.899811 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:15.899908 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:15.899998 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:15.900057 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:15.900129 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:15.900178 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:15.900263 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:15.900329 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:15.900393 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:15.900885 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:17:15.901106 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.901217 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.901366 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0724 17:17:15.901414 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0724 17:17:15.901454 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:15.901529 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:15.901467 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0724 17:17:15.901632 29520 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" } ], Type: "network" } I0724 17:17:15.901650 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0724 17:17:15.901773 29520 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0724 17:17:15.901861 29520 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0724 17:17:15.901915 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0724 17:17:15.901932 29520 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:15.902025 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0724 17:17:15.902250 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0724 17:17:15.902350 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.902423 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.902504 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:15.902568 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0724 17:17:15.902609 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0724 17:17:15.902734 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I0724 17:17:15.902887 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.903038 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0724 17:17:15.903227 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.903374 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.903473 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.903667 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0724 17:17:15.903772 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:15.903843 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.903930 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0724 17:17:15.904113 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.904190 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.904353 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I0724 17:17:15.904399 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0724 17:17:15.904497 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0724 17:17:15.904587 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0724 17:17:15.904669 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.904819 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0724 17:17:15.904853 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0724 17:17:15.904890 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-1", TransitGatewayId: "tgw-0123456" } I0724 17:17:15.904926 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.905080 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0724 17:17:15.905226 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" } ] }] } I0724 17:17:15.905372 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I0724 17:17:15.905515 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0724 17:17:15.905655 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.905800 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0724 17:17:15.905892 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:15.905936 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0724 17:17:15.905976 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.906019 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0724 17:17:15.906110 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0724 17:17:15.906261 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.906306 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0724 17:17:15.912710 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:17:15.912932 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.913109 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:15.913237 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0724 17:17:15.913282 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:15.913316 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.913369 29520 dns.go:238] Pre-creating DNS records I0724 17:17:15.913395 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:15.921840 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:15.925999 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.926105 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:15.965032 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.061177 29520 iamrole.go:89] ListRoles: { } I0724 17:17:16.063157 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run I0724 17:17:16.063579 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0724 17:17:16.063678 29520 api.go:53] PageSize not implemented I0724 17:17:16.063737 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.063915 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:16.063955 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:16.063951 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:16.065443 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.066082 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.066287 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:16.066452 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.066512 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:16.066633 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:16.066749 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0724 17:17:16.071641 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:17:16.071809 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:16.071896 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.071931 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.072049 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.072253 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:16.072302 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:16.072192 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:16.072268 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:16.072431 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:16.072455 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.072501 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.072620 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.072801 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:16.072845 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.072934 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0724 17:17:16.073074 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:16.073242 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:16.073305 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0724 17:17:16.073447 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:16.073489 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:16.073630 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:16.073761 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.073985 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.074093 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0724 17:17:16.074236 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:16.076940 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:17:16.077248 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.077505 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.077640 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.077792 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:16.077904 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:16.077924 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.077975 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.078018 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.077974 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.078058 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.078162 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.078184 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.078220 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.078260 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.078444 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.078569 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.078727 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.079052 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.079288 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.079911 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:17:16.080063 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:16.080166 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:16.080194 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:16.080222 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.080377 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0724 17:17:16.086514 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.089950 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.101920 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:16.105755 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.105850 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.144576 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.250557 29520 iamrole.go:89] ListRoles: { } I0724 17:17:16.252867 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0724 17:17:16.253215 29520 api.go:53] PageSize not implemented I0724 17:17:16.253250 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:16.253323 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.253438 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:16.253592 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.253759 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:16.253867 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.253965 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:16.253991 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:16.254040 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.254074 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.254191 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:16.254287 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:16.255501 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:17:16.255637 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.255810 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:16.255827 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.255975 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:16.256059 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:16.256156 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.255976 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:16.256232 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:16.256293 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.256345 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:16.256437 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.256537 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.256621 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.256756 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:16.256914 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:16.256964 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.257231 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.257398 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:16.257462 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:16.257550 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.257699 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.257899 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:16.258135 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:16.258210 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.258406 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:16.261724 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:17:16.261951 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.262061 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.262076 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:16.262273 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:16.262240 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.262299 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.262361 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.262398 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.262437 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.262469 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.262500 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.262616 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.262694 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0724 17:17:16.262728 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.262759 29520 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0724 17:17:16.262789 29520 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0724 17:17:16.262794 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.262818 29520 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0724 17:17:16.262908 29520 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0724 17:17:16.262967 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.263002 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.263025 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.263363 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.263506 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.263750 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.263930 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.264034 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.265279 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:17:16.265392 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:16.265488 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:16.265512 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:16.265536 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.265632 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0724 17:17:16.265673 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.265703 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:16.265751 29520 dns.go:238] Pre-creating DNS records I0724 17:17:16.273339 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:16.277232 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.277331 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.316905 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.420110 29520 iamrole.go:89] ListRoles: { } I0724 17:17:16.422288 29520 executor.go:111] Tasks: 0 done / 116 total; 47 can run W0724 17:17:16.422549 29520 api.go:53] PageSize not implemented I0724 17:17:16.422476 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:16.422683 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:16.422888 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.422953 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:16.423068 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:16.423249 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:16.423365 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.423546 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.423682 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:16.423720 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:16.423780 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.423822 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.425015 29520 executor.go:111] Tasks: 47 done / 116 total; 27 can run I0724 17:17:16.425184 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:16.425171 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.425292 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:16.425333 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.425393 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.425418 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:16.425447 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:16.425473 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:16.425533 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.425496 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.425615 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.425707 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.427300 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:16.427379 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:16.427662 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.427901 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.428072 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.428316 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0724 17:17:16.428539 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:16.428780 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:16.428992 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.429177 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:16.429245 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:16.429369 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:16.429638 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:16.435610 29520 executor.go:111] Tasks: 74 done / 116 total; 39 can run I0724 17:17:16.435852 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.436033 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:16.436126 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:16.436147 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.436087 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.436201 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.436318 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.436356 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.436433 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.436473 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.436461 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.436505 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.436689 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.437099 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.437321 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.437455 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.437571 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.437683 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.437838 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.440203 29520 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0724 17:17:16.440356 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:16.440473 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:16.440500 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:16.440526 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.440631 29520 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0724 17:17:16.444945 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.448623 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.460696 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:16.464514 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.464610 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.506146 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.605643 29520 iamrole.go:89] ListRoles: { } I0724 17:17:16.607870 29520 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0724 17:17:16.608087 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:16.608367 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0724 17:17:16.608469 29520 api.go:53] PageSize not implemented I0724 17:17:16.608492 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:16.608528 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:16.608483 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:16.608568 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0724 17:17:16.608825 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.608895 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:16.609044 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.609196 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.609364 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0724 17:17:16.609516 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:16.609641 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:16.610729 29520 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0724 17:17:16.610876 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:16.610997 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.611047 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:16.610917 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:16.611285 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:16.611261 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:16.611341 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.611394 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:16.612905 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:16.612981 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.613129 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.613309 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.613451 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0724 17:17:16.613671 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.613855 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:16.614031 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.614197 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.614303 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.614505 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.614653 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.614818 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:16.614924 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:16.614993 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:16.615160 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:16.615277 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:16.620848 29520 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0724 17:17:16.621039 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.621142 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0724 17:17:16.621213 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0724 17:17:16.621247 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:16.621279 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.621360 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.621335 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.621428 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.621497 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.621479 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.621561 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.621602 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.621695 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.621923 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.622129 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.622225 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.622512 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.622614 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.622743 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.625269 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.625343 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.625516 29520 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0724 17:17:16.625649 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:16.625766 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:16.625792 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:16.625816 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.625915 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:17:16.625961 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.625991 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:16.626041 29520 dns.go:238] Pre-creating DNS records I0724 17:17:16.634818 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:16.638838 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.638941 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.681590 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.778493 29520 iamrole.go:89] ListRoles: { } I0724 17:17:16.780379 29520 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0724 17:17:16.780526 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:16.780644 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0724 17:17:16.780805 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:16.782941 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:16.783116 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.783397 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:16.783902 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:16.784101 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.784269 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W0724 17:17:16.784324 29520 api.go:53] PageSize not implemented I0724 17:17:16.784318 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:16.784444 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.784547 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.784753 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.788279 29520 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0724 17:17:16.788435 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.788524 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:16.788624 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:16.788576 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.788674 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:16.788750 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.788822 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:16.788789 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.788886 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.789003 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:16.789063 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:16.789252 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:16.789321 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:16.789465 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:16.789515 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.789596 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.789677 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:16.789789 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:16.789832 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.789902 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.789996 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0724 17:17:16.790090 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0724 17:17:16.790184 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.790252 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:16.790300 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:16.793890 29520 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0724 17:17:16.794223 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.794421 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.794492 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:16.794619 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:16.794654 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.794658 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.794750 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.794816 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.794857 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.794889 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.794919 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.795004 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.795303 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.795413 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.795600 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.795791 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.795979 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.796131 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.796371 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.798785 29520 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0724 17:17:16.798942 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:16.799054 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:16.799090 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:16.799127 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.799251 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0724 17:17:16.804468 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.807887 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.807984 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.818003 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:16.821892 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.821992 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.822077 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.862536 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.964325 29520 iamrole.go:89] ListRoles: { } I0724 17:17:16.966547 29520 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0724 17:17:16.966825 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:17:16.966922 29520 api.go:53] PageSize not implemented I0724 17:17:16.967107 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:16.967240 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.967510 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:16.967697 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0724 17:17:16.967866 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:16.968027 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.968269 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.968440 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:16.968491 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:16.968535 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.968692 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:16.968812 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.969372 29520 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0724 17:17:16.969511 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.969662 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:16.969622 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.969762 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:16.969849 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.969846 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:16.970025 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:16.970008 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:16.970009 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:16.970102 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.970200 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:16.970255 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.970396 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:16.970476 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.970571 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:16.970627 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.970770 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.970855 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.970999 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:16.971048 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:16.971186 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:16.971363 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:16.971461 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.971646 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:16.971693 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:16.975938 29520 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0724 17:17:16.976120 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.976287 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.976445 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.976457 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:16.976545 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:16.976563 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.976612 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.976658 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.976635 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.976695 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:16.976729 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:16.976761 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:16.976778 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.976935 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.977153 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:16.977341 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.977511 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:16.977806 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.978100 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.980490 29520 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0724 17:17:16.980584 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:16.980669 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:16.980693 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:16.980715 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:16.980923 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:17:16.980978 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:16.981015 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:16.981077 29520 dns.go:238] Pre-creating DNS records I0724 17:17:16.988893 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:16.992810 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.992909 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:16.992970 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.032294 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.134318 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.136586 29520 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0724 17:17:17.136947 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.137043 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:17.137156 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0724 17:17:17.137313 29520 api.go:53] PageSize not implemented I0724 17:17:17.137259 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:17.137398 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0724 17:17:17.137593 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.137804 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:17.137913 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:17.138013 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.138154 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.138260 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:17.138290 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:17.138344 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.140078 29520 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0724 17:17:17.140233 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:17.140385 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:17.140481 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:17.140261 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.140538 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:17.140503 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:17.140577 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.140656 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:17.140758 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.140962 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.142515 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:17.142818 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:17.142894 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:17.143460 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.143619 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0724 17:17:17.143838 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.143987 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.144182 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:17.144375 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0724 17:17:17.146762 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.146882 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:17.146935 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.146994 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:17.147041 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:17.147074 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:17.150853 29520 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0724 17:17:17.151045 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.151167 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.151275 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.151394 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:17.151500 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:17.151521 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.151471 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.151568 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.151604 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:17.151643 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:17.151681 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:17.151675 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.151715 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:17.151812 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.152073 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.152285 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.152563 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.152775 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.152944 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.155380 29520 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0724 17:17:17.155570 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:17.155676 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:17.155704 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:17.155731 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.155823 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0724 17:17:17.159062 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.162494 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.162585 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.162728 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.172296 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:17.176106 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.176200 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.176261 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.176313 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.215040 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.317706 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.319951 29520 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0724 17:17:17.320276 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0724 17:17:17.320495 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0724 17:17:17.320597 29520 api.go:53] PageSize not implemented I0724 17:17:17.320690 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.320814 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:17.320886 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:17.321062 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.321306 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:17.321427 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.321564 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:17.321669 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.321782 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:17.321818 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:17.321875 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.323395 29520 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0724 17:17:17.323704 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:17.323574 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.323807 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:17.323615 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:17.323900 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:17.323824 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:17.323981 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:17.324087 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:17.324104 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:17.324210 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.324444 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:17.324522 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.324744 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.324962 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:17.325028 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.325148 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:17.325343 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.325468 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0724 17:17:17.325648 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.325809 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.325934 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:17.326129 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.326241 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.326351 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:17.326407 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:17.328821 29520 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0724 17:17:17.328955 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.329092 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.329094 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:17.329216 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:17.329198 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.329244 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.329312 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.329381 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:17.329442 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:17.329374 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.329481 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:17.329622 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:17.329584 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.329939 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.330179 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.330528 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.330747 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.330944 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.331166 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.332813 29520 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0724 17:17:17.332919 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:17.333007 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:17.333032 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:17.333055 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.333176 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:17:17.333224 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.333258 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:17.333323 29520 dns.go:238] Pre-creating DNS records I0724 17:17:17.341186 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:17.345197 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.345300 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.345383 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.345457 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.384437 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.484660 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.486891 29520 executor.go:111] Tasks: 0 done / 117 total; 48 can run I0724 17:17:17.487158 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0724 17:17:17.487359 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.487536 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0724 17:17:17.487605 29520 api.go:53] PageSize not implemented I0724 17:17:17.487629 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.487698 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:17.487843 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0724 17:17:17.487974 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.488202 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0724 17:17:17.488343 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:17.488401 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:17.488503 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.488570 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0724 17:17:17.488653 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0724 17:17:17.490171 29520 executor.go:111] Tasks: 48 done / 117 total; 27 can run I0724 17:17:17.490339 29520 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:17.490379 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0724 17:17:17.490578 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:17.490631 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.490620 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0724 17:17:17.490786 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:17.490747 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0724 17:17:17.490875 29520 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0724 17:17:17.490976 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:17.490933 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:17.491092 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0724 17:17:17.491273 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.491382 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.491485 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.491592 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0724 17:17:17.491742 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.491921 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0724 17:17:17.491978 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0724 17:17:17.492117 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:17.492167 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.492284 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.492395 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.492685 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.492787 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:17.492854 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:17.495824 29520 executor.go:111] Tasks: 75 done / 117 total; 39 can run I0724 17:17:17.496064 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.496172 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:17.496256 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:17.496274 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.496334 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.496368 29520 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:17.496336 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.496404 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0724 17:17:17.496442 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0724 17:17:17.496489 29520 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0724 17:17:17.496461 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.496619 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.496789 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.496960 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.497264 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.497428 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.497586 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.498653 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:17.498824 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.504763 29520 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0724 17:17:17.504904 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:17.505046 29520 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0724 17:17:17.505083 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0724 17:17:17.505126 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.505214 29520 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0724 17:17:17.505528 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:17.505586 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:17.505622 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:17.505655 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:17.505696 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:17.505733 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:17.505772 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:17.505808 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:17.505845 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:17.505887 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:17.505931 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:17.505964 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:17.505995 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:17.506031 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:17.506064 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:17.506101 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:17.506139 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:17.506176 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:17.506213 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:17.506253 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:17.506287 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:17.506328 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0724 17:17:17.506369 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:17.506405 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:17.506439 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:17.506477 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:17.506518 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:17.506556 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:17.509782 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:17.509817 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:17.509852 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.509912 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0724 17:17:17.509982 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510042 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510090 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510146 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510188 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510234 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0724 17:17:17.510290 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510385 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0724 17:17:17.510431 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0724 17:17:17.510489 29520 api.go:53] PageSize not implemented I0724 17:17:17.510508 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:17.510535 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:17.510548 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0724 17:17:17.510588 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:17.510612 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:17.510629 29520 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I0724 17:17:17.510696 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.510723 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:17.510765 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:17.510810 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.510851 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:17.510901 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:17.510949 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } autoscaling-group:nodes.complex.example.com ok I0724 17:17:17.511755 29520 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0724 17:17:17.511753 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-config:lt-1 ok I0724 17:17:17.511841 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0724 17:17:17.511847 29520 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0724 17:17:17.511870 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.511921 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I0724 17:17:17.512023 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0724 17:17:17.511971 29520 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0724 17:17:17.511935 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok I0724 17:17:17.511965 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W0724 17:17:17.512208 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:17.512233 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:17.512283 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0724 17:17:17.512343 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:17.512398 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0724 17:17:17.512718 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:17.512738 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0724 17:17:17.512787 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0724 17:17:17.512874 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0724 17:17:17.512930 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:17.512978 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:17.513030 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:17.513064 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0724 17:17:17.513123 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0724 17:17:17.513220 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:17.513279 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:17.513270 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0724 17:17:17.513364 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0724 17:17:17.513399 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0724 17:17:17.513453 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0724 17:17:17.513522 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0724 17:17:17.513587 29520 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0724 17:17:17.513608 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0724 17:17:17.513678 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0724 17:17:17.513794 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0724 17:17:17.513912 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:17.514024 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:17.514333 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.81s) === RUN TestLifecycleExternalLB I0724 17:17:17.514658 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:17.514947 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:17.515010 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:17.515075 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:17.515152 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:17.515209 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:17.515264 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:17.515327 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:17.515372 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:17.515453 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0724 17:17:17.540382 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:17.543300 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.543426 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.584195 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.679980 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.681744 29520 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0724 17:17:17.682105 29520 api.go:53] PageSize not implemented I0724 17:17:17.682115 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0724 17:17:17.682231 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" }] } W0724 17:17:17.682317 29520 api.go:53] PageSize not implemented I0724 17:17:17.682294 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.682594 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:17.682346 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0724 17:17:17.682574 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:17.682786 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W0724 17:17:17.682780 29520 api.go:53] PageSize not implemented I0724 17:17:17.682833 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:17.682933 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:17.682898 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0724 17:17:17.682956 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.683008 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:17.683088 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.683040 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } I0724 17:17:17.683159 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.683292 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.683140 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.683395 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:17.683466 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:17.683477 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0724 17:17:17.683505 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.683748 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } I0724 17:17:17.683970 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:17.684116 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:17.684174 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.684340 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:17.684383 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:17.684411 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:17.684452 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.684501 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.684598 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:17.685284 29520 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0724 17:17:17.685489 29520 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0724 17:17:17.685458 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.685611 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.685628 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0724 17:17:17.685697 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.685820 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:17.685926 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0724 17:17:17.685929 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0724 17:17:17.685950 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0724 17:17:17.686050 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:17.686107 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:17.686197 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0724 17:17:17.686158 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0724 17:17:17.686241 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.686444 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0724 17:17:17.686440 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0724 17:17:17.686600 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0724 17:17:17.686656 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0724 17:17:17.686857 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.687035 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:17.687096 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:17.687149 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:17.687202 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:17.687251 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" }] }], VpcId: "vpc-1" } I0724 17:17:17.687352 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:17.687413 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:17.687462 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:17.687544 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:17.687587 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.687895 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0724 17:17:17.691910 29520 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0724 17:17:17.692038 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.692184 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.692376 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.692573 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.692696 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.692906 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.693112 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.693392 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.693577 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:17.693670 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0724 17:17:17.693840 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.693953 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:17.694004 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0724 17:17:17.694174 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.694341 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.694521 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0724 17:17:17.694689 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:17.694850 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I0724 17:17:17.695046 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0724 17:17:17.695217 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:17.695275 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0724 17:17:17.695318 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.695464 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0724 17:17:17.695603 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.695733 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.696847 29520 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0724 17:17:17.697148 29520 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0724 17:17:17.697206 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.697245 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:17.697302 29520 dns.go:238] Pre-creating DNS records I0724 17:17:17.697333 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:17.703195 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:17.706654 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.706766 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.746036 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.845175 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.846882 29520 executor.go:111] Tasks: 0 done / 92 total; 50 can run W0724 17:17:17.847104 29520 api.go:53] PageSize not implemented W0724 17:17:17.847292 29520 api.go:53] PageSize not implemented I0724 17:17:17.847346 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.847450 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.847435 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0724 17:17:17.847608 29520 api.go:53] PageSize not implemented I0724 17:17:17.847593 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0724 17:17:17.847603 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.847737 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0724 17:17:17.847851 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:17.847932 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:17.847951 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:17.848027 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:17.848097 29520 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.848104 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.848188 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:17.848250 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.848390 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.848490 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:17.848522 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:17.848569 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.850374 29520 executor.go:111] Tasks: 50 done / 92 total; 19 can run I0724 17:17:17.850528 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0724 17:17:17.850533 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] }] } I0724 17:17:17.850807 29520 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0724 17:17:17.850728 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.850922 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.851062 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.851163 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:17.851211 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0724 17:17:17.851383 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.851446 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:17.851584 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0724 17:17:17.851782 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:17.860082 29520 executor.go:111] Tasks: 69 done / 92 total; 21 can run I0724 17:17:17.860287 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.860559 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.860746 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.860942 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.861276 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.861477 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.861659 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:17.864111 29520 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0724 17:17:17.864331 29520 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0724 17:17:17.867743 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:17.867803 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:17.867838 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:17.867870 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:17.867911 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:17.867945 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:17.867975 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:17.868008 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:17.868041 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:17.868075 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:17.868119 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:17.868160 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:17.868193 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:17.868224 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:17.868260 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:17.868293 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:17.868327 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:17.868360 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:17.868395 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:17.868428 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:17.868459 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:17.868491 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:17.868526 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:17.868562 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:17.868595 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:17.869963 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:17.869989 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:17.870025 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870074 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0724 17:17:17.870131 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870187 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870232 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870286 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870351 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870391 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0724 17:17:17.870437 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870486 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0724 17:17:17.870525 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0724 17:17:17.870577 29520 api.go:53] PageSize not implemented I0724 17:17:17.870597 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:17.870623 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:17.870639 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:17.870661 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:17.870678 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0724 17:17:17.870726 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:17.870752 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:17.870800 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:17.870834 29520 iamrole.go:89] ListRoles: { } I0724 17:17:17.870874 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:17.870915 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:17.870950 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-1 ok I0724 17:17:17.871420 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0724 17:17:17.871478 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0724 17:17:17.871442 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:17.871530 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0724 17:17:17.871594 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0724 17:17:17.871582 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.externallb.example.com ok I0724 17:17:17.871509 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I0724 17:17:17.871674 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0724 17:17:17.871920 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:17.871945 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:17.871997 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0724 17:17:17.872057 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0724 17:17:17.872118 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0724 17:17:17.872186 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:17.872234 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0724 17:17:17.872347 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:17.872368 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:17.872417 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:17.872518 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0724 17:17:17.872570 29520 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0724 17:17:17.872571 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:17.872594 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0724 17:17:17.872641 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0724 17:17:17.872680 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0724 17:17:17.872763 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0724 17:17:17.872778 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0724 17:17:17.872811 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0724 17:17:17.872892 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:17.872982 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:17.873060 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:17.873305 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.36s) === RUN TestLifecycleSharedSubnet I0724 17:17:17.873573 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:17.873876 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:17.873921 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:17.873961 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:17.874007 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:17.874045 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:17.874073 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:17.874108 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:17.874135 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:17.874185 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0724 17:17:17.898124 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:17.900923 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.901015 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:17.939688 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.040986 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.042486 29520 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0724 17:17:18.042859 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.042984 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0724 17:17:18.043039 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:18.043104 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0724 17:17:18.043134 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:18.043308 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:18.043370 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.043529 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:18.043673 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:18.043726 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.043778 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.043762 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0724 17:17:18.043933 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.043935 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0724 17:17:18.044104 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:18.044180 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:18.044208 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:18.045601 29520 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0724 17:17:18.045737 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0724 17:17:18.045902 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0724 17:17:18.045989 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0724 17:17:18.045982 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:18.046125 29520 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0724 17:17:18.046144 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.046272 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:18.046338 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.046308 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0724 17:17:18.046424 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0724 17:17:18.046659 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0724 17:17:18.046711 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0724 17:17:18.046897 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:18.046967 29520 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" } ] } I0724 17:17:18.047104 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.047193 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:18.047243 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.048748 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0724 17:17:18.050679 29520 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0724 17:17:18.050896 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" }] }] } I0724 17:17:18.051159 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.051325 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.051468 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.051744 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.051964 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0724 17:17:18.052117 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.052250 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0724 17:17:18.052383 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.052488 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.052655 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.052843 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" }] }] } I0724 17:17:18.052996 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.053149 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0724 17:17:18.053283 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.053408 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.053548 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0724 17:17:18.054284 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" }] }] } I0724 17:17:18.055124 29520 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0724 17:17:18.055527 29520 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0724 17:17:18.055631 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.055718 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.055828 29520 dns.go:238] Pre-creating DNS records I0724 17:17:18.055875 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.064944 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:18.068430 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.068528 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.107592 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.199907 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.201349 29520 executor.go:111] Tasks: 0 done / 78 total; 42 can run I0724 17:17:18.201818 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:18.201977 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:18.202035 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:18.202108 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.202356 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.202336 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:18.202480 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.202673 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.205935 29520 executor.go:111] Tasks: 42 done / 78 total; 16 can run I0724 17:17:18.206266 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0724 17:17:18.206346 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0724 17:17:18.206482 29520 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0724 17:17:18.206695 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0724 17:17:18.207143 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.207335 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0724 17:17:18.208208 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.208311 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:18.214180 29520 executor.go:111] Tasks: 58 done / 78 total; 18 can run I0724 17:17:18.214455 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.214889 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.215106 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.215453 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:18.217704 29520 executor.go:111] Tasks: 76 done / 78 total; 2 can run I0724 17:17:18.217947 29520 executor.go:111] Tasks: 78 done / 78 total; 0 can run I0724 17:17:18.221205 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:18.221270 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:18.221306 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:18.221338 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:18.221373 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:18.221407 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:18.221440 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:18.221477 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:18.221517 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:18.221561 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:18.221598 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:18.221631 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:18.221661 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:18.221697 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:18.221732 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:18.221767 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:18.221800 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:18.221833 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:18.221869 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:18.221908 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:18.223417 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:18.223448 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:18.223487 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223539 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0724 17:17:18.223596 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223670 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223716 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223759 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223802 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223838 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0724 17:17:18.223877 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223921 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0724 17:17:18.223955 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0724 17:17:18.224004 29520 api.go:53] PageSize not implemented I0724 17:17:18.224025 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:18.224055 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:18.224073 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:18.224097 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:18.224115 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0724 17:17:18.224166 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.224195 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.224244 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:18.224277 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.224326 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:18.224365 29520 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok autoscaling-group:nodes.sharedsubnet.example.com ok I0724 17:17:18.224767 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0724 17:17:18.224832 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.224819 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0724 17:17:18.224905 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0724 17:17:18.224949 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0724 17:17:18.224888 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0724 17:17:18.225002 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok W0724 17:17:18.225017 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.225034 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:18.224861 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I0724 17:17:18.225070 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.225120 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0724 17:17:18.225315 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.225335 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0724 17:17:18.225398 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:18.225465 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0724 17:17:18.225584 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0724 17:17:18.225636 29520 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0724 17:17:18.225653 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0724 17:17:18.225687 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0724 17:17:18.225731 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0724 17:17:18.225811 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0724 17:17:18.225826 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0724 17:17:18.225859 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0724 17:17:18.226103 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.35s) === RUN TestLifecyclePrivateSharedSubnet I0724 17:17:18.226380 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:18.226637 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:18.226686 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:18.226731 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:18.226779 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:18.226820 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:18.226850 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:18.226889 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:18.226918 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:18.226988 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0724 17:17:18.254825 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:18.257901 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.257994 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.258056 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.298374 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.400991 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.402831 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0724 17:17:18.403129 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.403201 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.403358 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.403514 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.403578 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:18.403722 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:18.403771 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:18.403710 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0724 17:17:18.403779 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:18.403851 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.404060 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.404087 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0724 17:17:18.404264 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0724 17:17:18.404321 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:18.404456 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0724 17:17:18.404552 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:18.404622 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0724 17:17:18.404680 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:18.404884 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:18.405567 29520 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0724 17:17:18.405715 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0724 17:17:18.405877 29520 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0724 17:17:18.405919 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.405930 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0724 17:17:18.406090 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0724 17:17:18.406170 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0724 17:17:18.406164 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.406263 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:18.406312 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.406321 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0724 17:17:18.406409 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0724 17:17:18.406356 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.406463 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:18.406478 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0724 17:17:18.406515 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.406576 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0724 17:17:18.406741 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0724 17:17:18.406792 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.406901 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0724 17:17:18.407019 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:18.407125 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0724 17:17:18.407196 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0724 17:17:18.407309 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0724 17:17:18.407416 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:18.407467 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:18.407506 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.407601 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:18.407641 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:18.407676 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:18.407724 29520 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Utility" },{ Key: "kubernetes.io/role/elb", Value: "1" }] } I0724 17:17:18.407807 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.408507 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:18.408717 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:18.409141 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:18.409283 29520 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" },{ Key: "kubernetes.io/role/internal-elb", Value: "1" }] } I0724 17:17:18.409896 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.414558 29520 executor.go:111] Tasks: 69 done / 104 total; 31 can run I0724 17:17:18.414791 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } W0724 17:17:18.415113 29520 api.go:53] PageSize not implemented W0724 17:17:18.415171 29520 api.go:53] PageSize not implemented I0724 17:17:18.415044 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.415327 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0724 17:17:18.415346 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0724 17:17:18.415444 29520 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0724 17:17:18.415527 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0724 17:17:18.415573 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0724 17:17:18.415554 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0724 17:17:18.415640 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0724 17:17:18.415808 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:18.415844 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.415942 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.416000 29520 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0724 17:17:18.416101 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.416145 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0724 17:17:18.416205 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0724 17:17:18.416055 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.416315 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.416463 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.416613 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0724 17:17:18.416712 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.416854 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.417030 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.417198 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" }] }] } I0724 17:17:18.417329 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.417379 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0724 17:17:18.417510 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.417642 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0724 17:17:18.417775 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0724 17:17:18.417902 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.417977 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.418070 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0724 17:17:18.418197 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" }] }] } I0724 17:17:18.418337 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:18.418445 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0724 17:17:18.418568 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" }] }] } I0724 17:17:18.418701 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0724 17:17:18.418831 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:18.418955 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.419087 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.420035 29520 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0724 17:17:18.420224 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:18.420406 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:18.420527 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:17:18.420567 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.420599 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.420647 29520 dns.go:238] Pre-creating DNS records I0724 17:17:18.420671 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.427508 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:18.430530 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.430623 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.430686 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.471385 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.572227 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.574254 29520 executor.go:111] Tasks: 0 done / 104 total; 45 can run I0724 17:17:18.574730 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:18.574766 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.575070 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:18.575180 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:18.575232 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.575494 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:18.575607 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.575707 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.576914 29520 executor.go:111] Tasks: 45 done / 104 total; 24 can run I0724 17:17:18.577073 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0724 17:17:18.577193 29520 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0724 17:17:18.577247 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0724 17:17:18.577130 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:18.577324 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0724 17:17:18.577549 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.577611 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0724 17:17:18.577809 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:18.577871 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0724 17:17:18.578017 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:18.578068 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0724 17:17:18.578211 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:18.578261 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0724 17:17:18.578420 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0724 17:17:18.578527 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0724 17:17:18.578699 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.582176 29520 executor.go:111] Tasks: 69 done / 104 total; 31 can run W0724 17:17:18.582440 29520 api.go:53] PageSize not implemented I0724 17:17:18.582419 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.582479 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.582581 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0724 17:17:18.582622 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } W0724 17:17:18.582705 29520 api.go:53] PageSize not implemented I0724 17:17:18.582641 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.582727 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.582773 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.582831 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.582804 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.582990 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0724 17:17:18.583396 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.583561 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.585682 29520 executor.go:111] Tasks: 100 done / 104 total; 4 can run W0724 17:17:18.585796 29520 api.go:53] PageSize not implemented I0724 17:17:18.585800 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:18.585821 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.585887 29520 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0724 17:17:18.585915 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0724 17:17:18.586000 29520 api.go:53] PageSize not implemented I0724 17:17:18.586034 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0724 17:17:18.586217 29520 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0724 17:17:18.589671 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:18.589737 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:18.589774 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:18.589807 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:18.589845 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:18.589878 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:18.589911 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:18.589964 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:18.590005 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:18.590043 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0724 17:17:18.590082 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:18.590116 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:18.590147 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:18.590179 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:18.590215 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:18.590251 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:18.590289 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:18.590351 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:18.590388 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:18.590425 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:18.590462 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:18.590496 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:18.590532 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:18.590568 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:18.592174 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:18.592206 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:18.592247 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592313 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0724 17:17:18.592399 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592459 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592503 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592547 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592589 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592626 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0724 17:17:18.592664 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592708 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0724 17:17:18.592745 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0724 17:17:18.592791 29520 api.go:53] PageSize not implemented I0724 17:17:18.592809 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0724 17:17:18.592862 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:18.592890 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:18.592908 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:18.592932 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:18.592950 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:18.593003 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.593029 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.593076 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:18.593115 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.593159 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:18.593201 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:18.593746 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.593793 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok autoscaling-group:nodes.private-shared-subnet.example.com ok I0724 17:17:18.593865 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0724 17:17:18.593967 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0724 17:17:18.593815 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0724 17:17:18.594027 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.594048 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:18.593961 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0724 17:17:18.594086 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:18.594116 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0724 17:17:18.594159 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0724 17:17:18.594150 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:bastions.private-shared-subnet.example.com ok volume:vol-2 ok autoscaling-config:lt-2 ok I0724 17:17:18.594056 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0724 17:17:18.594217 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:18.594223 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.594326 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.594277 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } iam-instance-profile:masters.private-shared-subnet.example.com ok W0724 17:17:18.594364 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.594387 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0724 17:17:18.594426 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.594505 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0724 17:17:18.594652 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.594677 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0724 17:17:18.594753 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-3 ok I0724 17:17:18.594892 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0724 17:17:18.594947 29520 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0724 17:17:18.594970 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0724 17:17:18.594946 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:18.595008 29520 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0724 17:17:18.595030 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0724 17:17:18.595132 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.595148 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0724 17:17:18.595059 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0724 17:17:18.595188 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:18.595216 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0724 17:17:18.595235 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0724 17:17:18.595310 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0724 17:17:18.595350 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok W0724 17:17:18.595332 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.595395 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0724 17:17:18.595399 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.595487 29520 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0724 17:17:18.595503 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0724 17:17:18.595538 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0724 17:17:18.595800 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedIP I0724 17:17:18.596078 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:18.596330 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:18.596378 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:18.596420 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:18.596465 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:18.596503 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:18.596537 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:18.596572 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:18.596599 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:18.596652 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0724 17:17:18.624342 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:18.630961 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.631103 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.631210 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.673915 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.771124 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.773249 29520 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0724 17:17:18.773686 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.773788 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0724 17:17:18.773808 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.773925 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0724 17:17:18.773993 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:18.774099 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0724 17:17:18.776890 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0724 17:17:18.777130 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0724 17:17:18.774113 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.777312 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0724 17:17:18.777548 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I0724 17:17:18.777834 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.778167 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:18.778367 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:18.778514 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:18.778760 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:18.780854 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:18.780971 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:18.781199 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:18.781755 29520 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0724 17:17:18.781907 29520 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0724 17:17:18.781900 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.782013 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0724 17:17:18.782159 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0724 17:17:18.782099 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.782241 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0724 17:17:18.782183 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0724 17:17:18.782247 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0724 17:17:18.782379 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0724 17:17:18.782348 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0724 17:17:18.782501 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:18.782530 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0724 17:17:18.782656 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.782781 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0724 17:17:18.782866 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0724 17:17:18.782919 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0724 17:17:18.782913 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0724 17:17:18.783050 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:18.783134 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" } ] } I0724 17:17:18.783255 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0724 17:17:18.783289 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.783408 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:18.783467 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.783510 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0724 17:17:18.783622 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0724 17:17:18.783754 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0724 17:17:18.783900 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.784027 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.784146 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.784237 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:18.784282 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:18.784316 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.784419 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:18.784463 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:18.784498 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:18.784544 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:18.784578 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:18.784629 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:18.784664 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0724 17:17:18.784356 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0724 17:17:18.784750 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:18.784857 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:18.784911 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:18.784969 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.785008 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:18.785144 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0724 17:17:18.785244 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:18.785289 29520 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0724 17:17:18.787241 29520 executor.go:111] Tasks: 71 done / 113 total; 36 can run W0724 17:17:18.787601 29520 api.go:53] PageSize not implemented I0724 17:17:18.787496 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.787747 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0724 17:17:18.787856 29520 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0724 17:17:18.787752 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" }] }] } W0724 17:17:18.787934 29520 api.go:53] PageSize not implemented I0724 17:17:18.787976 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0724 17:17:18.788082 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0724 17:17:18.788151 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0724 17:17:18.788044 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0724 17:17:18.788218 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0724 17:17:18.788318 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0724 17:17:18.788235 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0724 17:17:18.788368 29520 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" }] } I0724 17:17:18.788433 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0724 17:17:18.788464 29520 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0724 17:17:18.788512 29520 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0724 17:17:18.788468 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.788639 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I0724 17:17:18.788881 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.788978 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I0724 17:17:18.789166 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0724 17:17:18.789309 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:18.789448 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:18.789497 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:18.789569 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:18.789645 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0724 17:17:18.789748 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:18.789827 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0724 17:17:18.789978 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.790103 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.790294 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.790431 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.790537 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:18.790678 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0724 17:17:18.790729 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.790863 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.791004 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:18.791140 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0724 17:17:18.791225 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0724 17:17:18.791273 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0724 17:17:18.791442 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" }] }] } I0724 17:17:18.791606 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" }] }] } I0724 17:17:18.791744 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0724 17:17:18.791843 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0724 17:17:18.791976 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.792109 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.792244 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0724 17:17:18.792314 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:18.792439 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0724 17:17:18.792588 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0724 17:17:18.792705 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.792821 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.792889 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.793081 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0724 17:17:18.793128 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:18.793193 29520 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0724 17:17:18.795233 29520 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0724 17:17:18.795462 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:18.795478 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:18.795588 29520 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:18.795692 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:18.795721 29520 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0724 17:17:18.795848 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0724 17:17:18.795904 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:18.795946 29520 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0724 17:17:18.796027 29520 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0724 17:17:18.796097 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:18.796225 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0724 17:17:18.796295 29520 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0724 17:17:18.796338 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.796381 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.796431 29520 dns.go:238] Pre-creating DNS records I0724 17:17:18.796455 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.802802 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.13 ********************************************************************************* I0724 17:17:18.805838 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.805928 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.805994 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.844437 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.947027 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.949275 29520 executor.go:111] Tasks: 0 done / 113 total; 45 can run I0724 17:17:18.949488 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.949679 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0724 17:17:18.949755 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0724 17:17:18.949783 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0724 17:17:18.949830 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.949837 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:18.949893 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:18.950182 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.952359 29520 executor.go:111] Tasks: 45 done / 113 total; 26 can run I0724 17:17:18.952556 29520 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0724 17:17:18.952537 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0724 17:17:18.952751 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.952839 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0724 17:17:18.952882 29520 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0724 17:17:18.952877 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0724 17:17:18.953087 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:18.953154 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.953298 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0724 17:17:18.953346 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0724 17:17:18.953555 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:18.953602 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0724 17:17:18.953836 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:18.953899 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0724 17:17:18.954011 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.954067 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.954224 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:18.954266 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0724 17:17:18.954413 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.954506 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.957713 29520 executor.go:111] Tasks: 71 done / 113 total; 36 can run I0724 17:17:18.957889 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0724 17:17:18.957986 29520 api.go:53] PageSize not implemented W0724 17:17:18.958021 29520 api.go:53] PageSize not implemented I0724 17:17:18.958039 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.958058 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0724 17:17:18.958177 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0724 17:17:18.958140 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:18.958238 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0724 17:17:18.958292 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.958344 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0724 17:17:18.958445 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0724 17:17:18.958436 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.958481 29520 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0724 17:17:18.958524 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0724 17:17:18.958643 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:18.958778 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:18.959004 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.959113 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:18.959202 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0724 17:17:18.959434 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0724 17:17:18.961684 29520 executor.go:111] Tasks: 107 done / 113 total; 5 can run I0724 17:17:18.961780 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:18.961861 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0724 17:17:18.961878 29520 api.go:53] PageSize not implemented I0724 17:17:18.961864 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0724 17:17:18.961955 29520 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0724 17:17:18.961913 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0724 17:17:18.962122 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0724 17:17:18.962201 29520 api.go:53] PageSize not implemented I0724 17:17:18.962233 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0724 17:17:18.962435 29520 executor.go:111] Tasks: 112 done / 113 total; 1 can run I0724 17:17:18.962509 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:18.962617 29520 executor.go:111] Tasks: 113 done / 113 total; 0 can run I0724 17:17:18.966143 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:18.966205 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:18.966244 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:18.966277 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:18.966317 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:18.966351 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:18.966385 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:18.966422 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:18.966462 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:18.966501 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0724 17:17:18.966544 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0724 17:17:18.966581 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:18.966614 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:18.966647 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:18.966683 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:18.966718 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0724 17:17:18.966754 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:18.966791 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:18.966826 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0724 17:17:18.966862 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0724 17:17:18.966899 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0724 17:17:18.966936 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:18.966993 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:18.967031 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0724 17:17:18.967070 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:18.967105 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:18.967137 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:18.967176 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:18.967213 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:18.968768 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:18.968796 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:18.968836 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.968908 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0724 17:17:18.968992 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969054 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969099 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969140 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969182 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969224 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0724 17:17:18.969278 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969333 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:18.969361 29520 natgateway.go:117] DescribeNatGateways: { } I0724 17:17:18.969406 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0724 17:17:18.969443 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0724 17:17:18.969488 29520 api.go:53] PageSize not implemented I0724 17:17:18.969505 29520 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0724 17:17:18.969560 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:18.969587 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:18.969607 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:18.969629 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:18.969649 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:18.969703 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:18.969730 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.969773 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:18.969812 29520 iamrole.go:89] ListRoles: { } I0724 17:17:18.969862 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:18.969911 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:18.969945 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0724 17:17:18.969982 29520 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0724 17:17:18.970016 29520 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:bastion.private-shared-ip.example.com ok I0724 17:17:18.971799 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:18.971779 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:18.971947 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0724 17:17:18.971902 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0724 17:17:18.972086 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.972109 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } autoscaling-group:nodes.private-shared-ip.example.com ok I0724 17:17:18.971934 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0724 17:17:18.972145 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:18.972187 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-ip.example.com. ok I0724 17:17:18.972343 29520 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok load-balancer:bastion-private-shared-ip-eepmph ok I0724 17:17:18.972215 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0724 17:17:18.972448 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.972361 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } security-group:sg-2 ok I0724 17:17:18.972496 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0724 17:17:18.972487 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0724 17:17:18.972579 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0724 17:17:18.972683 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0724 17:17:18.972644 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0724 17:17:18.972829 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.972884 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-2 ok I0724 17:17:18.973018 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:18.973143 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0724 17:17:18.973303 29520 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-1 ok I0724 17:17:18.973474 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0724 17:17:18.973557 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:18.973827 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0724 17:17:18.973853 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0724 17:17:18.973954 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0724 17:17:18.974000 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0724 17:17:18.974189 29520 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0724 17:17:18.973970 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0724 17:17:18.974557 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.974579 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0724 17:17:18.974625 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0724 17:17:18.974695 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:18.974740 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0724 17:17:18.974502 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0724 17:17:18.974793 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0724 17:17:18.976124 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:18.976102 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0724 17:17:18.976140 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0724 17:17:18.976168 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0724 17:17:18.976238 29520 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0724 17:17:18.976253 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0724 17:17:18.976285 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0724 17:17:18.976319 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0724 17:17:18.976381 29520 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0724 17:17:18.976475 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:18.976553 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0724 17:17:18.976833 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.38s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0724 17:17:18.977182 29520 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:17:18.977476 29520 internetgateways.go:66] CreateInternetGateway: { } I0724 17:17:18.977526 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:17:18.977568 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:17:18.977615 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:17:18.977657 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:17:18.977686 29520 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:17:18.977720 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:17:18.977748 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:17:18.977804 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0724 17:17:19.003302 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0724 17:17:19.006218 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.006316 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.052384 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:19.175674 29520 iamrole.go:89] ListRoles: { } I0724 17:17:19.177425 29520 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0724 17:17:19.177823 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.178012 29520 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.178044 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0724 17:17:19.178108 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:19.178066 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:19.178126 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0724 17:17:19.178149 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.178253 29520 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0724 17:17:19.178262 29520 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.178333 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0724 17:17:19.178320 29520 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0724 17:17:19.178358 29520 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0724 17:17:19.178463 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:19.178601 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:19.178670 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.178931 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:19.179087 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:19.179145 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.179312 29520 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0724 17:17:19.179442 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:19.179491 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:19.179543 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:19.179687 29520 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.180462 29520 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0724 17:17:19.180604 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0724 17:17:19.180679 29520 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0724 17:17:19.180721 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.180906 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:19.181025 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0724 17:17:19.181063 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0724 17:17:19.181193 29520 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0724 17:17:19.181377 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.181439 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0724 17:17:19.181526 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0724 17:17:19.181693 29520 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0724 17:17:19.181691 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.181995 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:19.181960 29520 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0724 17:17:19.182192 29520 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:17:19.182254 29520 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.182383 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:19.182483 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:19.182547 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:19.182601 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0724 17:17:19.182745 29520 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0724 17:17:19.182808 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I0724 17:17:19.182991 29520 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0724 17:17:19.183101 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:19.183161 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:19.183212 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:19.183264 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:19.183302 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:19.183347 29520 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0724 17:17:19.183719 29520 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0724 17:17:19.185535 29520 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0724 17:17:19.185744 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.186120 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.186403 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.186666 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.186929 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.187136 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.187269 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.187356 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.187455 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.187572 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.187752 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.187917 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.188088 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0724 17:17:19.188159 29520 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0724 17:17:19.188221 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.188385 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.188556 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0724 17:17:19.188729 29520 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.188890 29520 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:19.188970 29520 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0724 17:17:19.189042 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.189203 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.189334 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.189397 29520 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0724 17:17:19.190325 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.196934 29520 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0724 17:17:19.197267 29520 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0724 17:17:19.197571 29520 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0724 17:17:19.197628 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:19.197666 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:19.197726 29520 dns.go:238] Pre-creating DNS records I0724 17:17:19.197757 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:19.203496 29520 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.0 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.0 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0724 17:17:19.206676 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.206770 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.248315 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:19.375206 29520 iamrole.go:89] ListRoles: { } I0724 17:17:19.377071 29520 executor.go:111] Tasks: 0 done / 99 total; 45 can run I0724 17:17:19.377736 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.377866 29520 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0724 17:17:19.378010 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.378222 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.378340 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:17:19.378375 29520 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:17:19.378426 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:19.378475 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.378695 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:19.378775 29520 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0724 17:17:19.379768 29520 executor.go:111] Tasks: 45 done / 99 total; 24 can run I0724 17:17:19.379966 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:19.380089 29520 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0724 17:17:19.380142 29520 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0724 17:17:19.380081 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.380228 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0724 17:17:19.380458 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:19.380525 29520 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:17:19.380626 29520 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:17:19.380675 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.380814 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.380945 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0724 17:17:19.381077 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:19.384920 29520 executor.go:111] Tasks: 69 done / 99 total; 26 can run I0724 17:17:19.385316 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.385453 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.385606 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.385699 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.385862 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.386039 29520 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0724 17:17:19.386222 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.388172 29520 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0724 17:17:19.388491 29520 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0724 17:17:19.388662 29520 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0724 17:17:19.391541 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0724 17:17:19.391608 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0724 17:17:19.391643 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0724 17:17:19.391677 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0724 17:17:19.391716 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0724 17:17:19.391750 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0724 17:17:19.391781 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0724 17:17:19.391814 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0724 17:17:19.391848 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0724 17:17:19.391882 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0724 17:17:19.391919 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0724 17:17:19.391963 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0724 17:17:19.391997 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0724 17:17:19.392030 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0724 17:17:19.392066 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0724 17:17:19.392099 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:17:19.392136 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0724 17:17:19.392172 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:17:19.392210 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0724 17:17:19.392243 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0724 17:17:19.392277 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0724 17:17:19.392309 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0724 17:17:19.392346 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0724 17:17:19.392384 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:17:19.392421 29520 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0724 17:17:19.393879 29520 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0724 17:17:19.393910 29520 keypairs.go:95] DescribeKeyPairs: { } I0724 17:17:19.393949 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394003 29520 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394066 29520 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394127 29520 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394176 29520 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394234 29520 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394277 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394318 29520 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394366 29520 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394422 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0724 17:17:19.394466 29520 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0724 17:17:19.394522 29520 api.go:53] PageSize not implemented I0724 17:17:19.394544 29520 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0724 17:17:19.394572 29520 loadbalancers.go:34] PageSize not implemented I0724 17:17:19.394592 29520 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0724 17:17:19.394616 29520 targetgroups.go:35] PageSize not implemented I0724 17:17:19.394634 29520 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0724 17:17:19.394686 29520 zones.go:93] ListHostedZonesPages { } I0724 17:17:19.394715 29520 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0724 17:17:19.394764 29520 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0724 17:17:19.394801 29520 iamrole.go:89] ListRoles: { } I0724 17:17:19.394849 29520 internetgateways.go:101] DescribeInternetGateways: { } I0724 17:17:19.394891 29520 routetable.go:53] DescribeRouteTables: { } I0724 17:17:19.394926 29520 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0724 17:17:19.395396 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0724 17:17:19.395462 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0724 17:17:19.395554 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0724 17:17:19.395591 29520 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0724 17:17:19.395639 29520 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0724 17:17:19.395498 29520 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-config:lt-1 ok I0724 17:17:19.395697 29520 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok internet-gateway:igw-2 ok I0724 17:17:19.395737 29520 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0724 17:17:19.395631 29520 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.nthsqsresources.longclustername.example.com. ok I0724 17:17:19.395834 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0724 17:17:19.395892 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0724 17:17:19.396026 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:19.396051 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0724 17:17:19.396087 29520 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0724 17:17:19.396146 29520 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0724 17:17:19.396191 29520 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0724 17:17:19.396405 29520 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0724 17:17:19.396418 29520 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0724 17:17:19.396514 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0724 17:17:19.396575 29520 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0724 17:17:19.396558 29520 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0724 17:17:19.396593 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0724 17:17:19.396655 29520 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0724 17:17:19.396699 29520 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0724 17:17:19.396753 29520 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0724 17:17:19.396768 29520 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0724 17:17:19.396800 29520 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0724 17:17:19.396878 29520 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0724 17:17:19.396965 29520 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0724 17:17:19.397069 29520 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0724 17:17:19.397381 29520 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 43.591s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.037s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0724 17:15:28.128061 5629 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.041s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.073s === RUN TestPodController I0724 17:16:05.494746 23896 pod.go:59] starting pod controller I0724 17:16:05.496370 23896 pod.go:65] shutting down pod controller I0724 17:16:05.496376 23896 pod.go:107] Got stop signal I0724 17:16:05.496338 23896 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.024s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.065s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a1dc0} route53_test.go:93: Got zone list: [0xc0000a1df0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e090} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a1ec0} route53_test.go:93: Got zone list: [0xc0000a1ef0] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e0a0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000a1f90} route53_test.go:170: Successfully added managed DNS zone: &{0xc0002c6ea0 0xc00000e0b8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e60f0} route53_test.go:93: Got zone list: [0xc0002e6120] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e0d0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e6220} route53_test.go:93: Got zone list: [0xc0002e6250] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e0f0} route53_test.go:187: Successfully added resource record set: {0xc000118100 0xc00000e110} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e64d0} route53_test.go:93: Got zone list: [0xc0002e6500] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e198} route53_test.go:199: Successfully added resource record set: {0xc000118280 0xc00000e1b8} route53_test.go:118: Got 1 recordsets: [0xc0002e66a0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e67d0} route53_test.go:93: Got zone list: [0xc0002e6800] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e250} route53_test.go:221: Successfully added resource record set: {0xc000118400 0xc00000e270} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000118400 0xc00000e270}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e6b60} route53_test.go:93: Got zone list: [0xc0002e6b90] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e328} route53_test.go:246: Successfully removed resource set {0xc000118600 0xc00000e348} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e6e10} route53_test.go:93: Got zone list: [0xc0002e6e40] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e3d0} route53_test.go:264: Successfully removed resource set {0xc000118780 0xc00000e3f0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e70f0} route53_test.go:93: Got zone list: [0xc0002e7120] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e480} commontests.go:192: Successfully added resource record set: {0xc000118900 0xc00000e4a0} commontests.go:45: Correctly replaced resource record {0xc000118900 0xc00000e4a0} -> {0xc000118a00 0xc00000e4f0} commontests.go:142: Got 1 recordsets: [0xc0002e7430] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e7600} route53_test.go:93: Got zone list: [0xc0002e7630] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e5a0} commontests.go:192: Successfully added resource record set: {0xc000118c80 0xc00000e5c0} commontests.go:70: Correctly replaced resource record {0xc000118c80 0xc00000e5c0} -> {0xc000118d80 0xc00000e610} commontests.go:142: Got 1 recordsets: [0xc0002e7940] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002e7b60} route53_test.go:93: Got zone list: [0xc0002e7b90] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e6d0} commontests.go:192: Successfully added resource record set: {0xc000119000 0xc00000e6f0} commontests.go:142: Got 2 recordsets: [0xc0002e7e00 0xc0002e7e20] commontests.go:142: Got 2 recordsets: [0xc0002e7e90 0xc0002e7eb0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0002f8050} route53_test.go:93: Got zone list: [0xc0002f8080] route53_test.go:98: Got at least 1 zone in list:&{0xc0002c6cc0 0xc00000e7f0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.010s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.018s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410020} clouddns_test.go:71: Got zone list: [0xc00059a0f0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059a090} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410160} clouddns_test.go:71: Got zone list: [0xc00059a1b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059a180} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410260} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0000dccc0 0xc00059a2d0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410320} clouddns_test.go:71: Got zone list: [0xc00059a438] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059a3d8} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410440} clouddns_test.go:71: Got zone list: [0xc00059a660] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059a618} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410580} clouddns_test.go:71: Got zone list: [0xc00059a900] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059a8a0} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc0004106e0} clouddns_test.go:71: Got zone list: [0xc00059acd8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059ac78} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410840} clouddns_test.go:71: Got zone list: [0xc00059af78] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059af18} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410980} clouddns_test.go:71: Got zone list: [0xc00059b218] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059b1b8} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410ae0} clouddns_test.go:71: Got zone list: [0xc00059b590] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059b518} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410c40} clouddns_test.go:71: Got zone list: [0xc00059ba40] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059b9e0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410dc0} clouddns_test.go:71: Got zone list: [0xc00059bec0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00059be90} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000880308 0xc000410fa0} clouddns_test.go:71: Got zone list: [0xc00037c528] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000732d80 0xc00037c4c8} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.051s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.070s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.065s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0724 17:16:36.065798 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.066091 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.066142 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.066191 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.066250 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.066292 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.066308 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.066340 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.066365 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.076765 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0724 17:16:36.080290 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.080453 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.080479 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.080504 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.080530 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.080549 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.080565 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.080586 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.080598 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.081612 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I0724 17:16:36.084899 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.085021 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.085044 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.085077 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.085102 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.085131 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.085150 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.085167 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.085180 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.086120 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.087495 29541 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0724 17:16:36.087513 29541 assetstore.go:111] bin/containerd I0724 17:16:36.087521 29541 assetstore.go:111] bin/containerd-shim I0724 17:16:36.087529 29541 assetstore.go:111] bin/containerd-shim-runc-v1 I0724 17:16:36.087536 29541 assetstore.go:111] bin/containerd-shim-runc-v2 I0724 17:16:36.087544 29541 assetstore.go:111] bin/containerd-stress I0724 17:16:36.087552 29541 assetstore.go:111] bin/ctr I0724 17:16:36.087589 29541 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0724 17:16:36.087603 29541 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0724 17:16:36.090574 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.090695 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.090728 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.090753 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.090778 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.090798 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.090826 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.090848 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.090861 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.091825 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.093059 29541 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0724 17:16:36.093970 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.094109 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.094132 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.094153 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.094176 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.094241 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.094257 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.094283 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.094312 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.095185 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.096939 29541 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0724 17:16:36.097593 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.097740 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.097767 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.097800 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.097829 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.097877 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.097893 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.097916 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.097930 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.098963 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.100410 29541 assetstore.go:108] Matching assets for "^docker/": I0724 17:16:36.100434 29541 assetstore.go:111] docker/containerd I0724 17:16:36.100443 29541 assetstore.go:111] docker/containerd-shim I0724 17:16:36.100451 29541 assetstore.go:111] docker/ctr I0724 17:16:36.100459 29541 assetstore.go:111] docker/docker I0724 17:16:36.100466 29541 assetstore.go:111] docker/docker-init I0724 17:16:36.100473 29541 assetstore.go:111] docker/docker-proxy I0724 17:16:36.100481 29541 assetstore.go:111] docker/dockerd I0724 17:16:36.100494 29541 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0724 17:16:36.103844 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.103982 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.104007 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.104030 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.104055 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.104098 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.104115 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.104136 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.104156 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.105044 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.106411 29541 assetstore.go:108] Matching assets for "^docker/": I0724 17:16:36.106428 29541 assetstore.go:111] docker/containerd I0724 17:16:36.106437 29541 assetstore.go:111] docker/containerd-shim I0724 17:16:36.106444 29541 assetstore.go:111] docker/ctr I0724 17:16:36.106451 29541 assetstore.go:111] docker/docker I0724 17:16:36.106458 29541 assetstore.go:111] docker/docker-init I0724 17:16:36.106465 29541 assetstore.go:111] docker/docker-proxy I0724 17:16:36.106473 29541 assetstore.go:111] docker/dockerd I0724 17:16:36.106487 29541 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0724 17:16:36.112080 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.112271 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.112310 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.112350 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.112381 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.112442 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.112461 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.112510 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.112528 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.113667 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.115337 29541 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0724 17:16:36.115534 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.115685 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.115713 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.115740 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.115768 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.115812 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.115844 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.115866 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.115880 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.116823 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:36.118319 29541 assetstore.go:108] Matching assets for "^docker/": I0724 17:16:36.118338 29541 assetstore.go:111] docker/docker I0724 17:16:36.118347 29541 assetstore.go:111] docker/docker-containerd I0724 17:16:36.118355 29541 assetstore.go:111] docker/docker-containerd-ctr I0724 17:16:36.118363 29541 assetstore.go:111] docker/docker-containerd-shim I0724 17:16:36.118370 29541 assetstore.go:111] docker/docker-init I0724 17:16:36.118378 29541 assetstore.go:111] docker/docker-proxy I0724 17:16:36.118398 29541 assetstore.go:111] docker/docker-runc I0724 17:16:36.118405 29541 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0724 17:16:36.122380 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.122512 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.122538 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.122559 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.122583 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.122604 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.122620 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.122642 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.122655 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.124813 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0724 17:16:36.127007 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.127138 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.127163 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.127186 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.127209 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.127230 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.127246 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.127268 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.127282 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.128924 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0724 17:16:36.131136 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.131261 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.131287 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.131313 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.131343 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.131365 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.131382 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.131401 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.131414 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.132908 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0724 17:16:36.138505 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.138634 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.138662 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.138684 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.138707 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.138728 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.138744 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.138764 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.138777 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.140272 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0724 17:16:36.152014 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.152274 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.152325 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.152376 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.152444 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.152487 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.152516 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.152549 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.152574 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.155295 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0724 17:16:36.165781 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.166013 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.166060 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.166101 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.166145 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.166180 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.166208 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.166243 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.166266 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.168830 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0724 17:16:36.178899 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.179133 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.179177 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.179217 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.179259 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.179296 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.179326 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.179360 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.179383 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.181874 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.02s) === RUN TestKubeAPIServerBuilderAMD64 I0724 17:16:36.197574 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:36.197741 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:36.197773 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:36.197803 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:36.197834 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:36.197858 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:36.197878 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:36.197900 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:36.197916 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:36.199836 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.43s) === RUN TestKubeAPIServerBuilderARM64 I0724 17:16:37.627589 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.627817 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.627871 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.627916 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.627964 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.628005 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.628030 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.628065 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.628091 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.630679 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0724 17:16:37.707070 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.707327 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.707380 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.707428 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.707479 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.707521 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.707552 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.707592 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.707618 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.710329 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.02s) === RUN TestKubeControllerManagerBuilderAMD64 I0724 17:16:37.723312 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.723570 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.723618 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.723684 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.723734 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.723772 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.723801 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.723858 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.723889 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.726664 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I0724 17:16:37.804123 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.804374 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.804427 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.804481 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.804532 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.804574 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.804606 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.804644 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.804672 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.807515 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0724 17:16:37.878084 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.878229 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.878259 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.878289 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.878329 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.878353 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.878374 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.878406 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.878421 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.880052 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0724 17:16:37.883954 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.884097 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.884129 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.884163 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.884189 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.884221 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.884244 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.884265 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.884279 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.885787 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0724 17:16:37.955880 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:37.956131 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:37.956184 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:37.956235 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:37.956286 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:37.956332 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:37.956363 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:37.956403 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:37.956431 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:37.959237 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0724 17:16:38.030272 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.030547 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.030586 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.030628 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.030660 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.030692 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.030711 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.030736 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.030756 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.032589 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0724 17:16:38.039711 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.039937 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.039986 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.040026 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.040070 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.040107 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.040137 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.040174 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.040197 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.042876 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0724 17:16:38.111837 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.112086 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.112128 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.112161 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.112197 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.112229 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.112248 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.112274 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.112294 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.114314 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.08s) === RUN TestKubectlBuilder I0724 17:16:38.191230 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.191417 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.191456 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.191487 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.191520 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.191547 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.191564 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.191586 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.191605 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.193301 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:38.195203 29541 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0724 17:16:38.196644 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.196769 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.196810 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.196835 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.196860 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.196881 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.196900 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.196923 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.196938 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.198251 29541 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0724 17:16:38.201776 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.201910 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.201951 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.201977 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.202005 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.202028 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.202046 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.202069 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.202084 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.203382 29541 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0724 17:16:38.206787 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.206920 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.206947 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.206983 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.207022 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.207048 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.207066 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.207085 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.207099 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.208669 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:38.210285 29541 assetstore.go:108] Matching assets for "protokube$": I0724 17:16:38.210304 29541 assetstore.go:111] /opt/kops/bin/protokube I0724 17:16:38.210334 29541 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0724 17:16:38.210358 29541 assetstore.go:108] Matching assets for "channels$": I0724 17:16:38.210369 29541 assetstore.go:111] /opt/kops/bin/channels I0724 17:16:38.210377 29541 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I0724 17:16:38.211762 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.211903 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.211931 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.211955 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.211980 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.212002 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.212019 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.212040 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.212054 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.213542 29541 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0724 17:16:38.215718 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.215844 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.215874 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.215900 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.215927 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.215949 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.215965 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.215986 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.216001 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.217544 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:38.219106 29541 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0724 17:16:38.219477 29541 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:38.219608 29541 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:38.219634 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:38.219659 29541 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:38.219686 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:38.219709 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:38.219728 29541 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:38.219749 29541 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:38.219762 29541 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:38.224312 29541 zones.go:93] ListHostedZonesPages { } I0724 17:16:38.226731 29541 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 2.228s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.069s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.088s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.063s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.042s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0724 17:15:48.768509 16047 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0724 17:15:48.768659 16047 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.010s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0724 17:16:24.993420 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.993531 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.993587 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.993633 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.993705 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0724 17:16:24.994004 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994056 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994098 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994138 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994186 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994226 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994277 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994318 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994363 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994409 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994447 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994494 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994558 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994604 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994642 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994698 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994741 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994780 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0724 17:16:24.994817 28587 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0724 17:16:24.995033 28587 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0724 17:16:24.995110 28587 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.083s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0724 17:16:01.798287 22386 builder.go:215] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.01s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0724 17:16:01.799271 22386 builder.go:215] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0724 17:16:01.800206 22386 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0724 17:16:01.804512 22386 builder.go:215] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0724 17:16:01.805492 22386 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0724 17:16:01.808470 22386 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0724 17:16:01.811493 22386 builder.go:215] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.01s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0724 17:16:01.812965 22386 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0724 17:16:01.812983 22386 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0724 17:16:01.812992 22386 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0724 17:16:01.813003 22386 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.094s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.067s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.077s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.097s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.101s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.073s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.064s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.070s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.036s === RUN TestFlagToFalse I0724 17:15:27.803909 5143 featureflag_test.go:33] Created flag Unittest1 I0724 17:15:27.804003 5143 featureflag.go:162] FeatureFlag "UnitTest1"=false I0724 17:15:27.804028 5143 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0724 17:15:27.804106 5143 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.083s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.037s === RUN TestRenderInstanceGroupZones W0724 17:16:02.121922 22641 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.123s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0724 17:16:44.989211 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:44.989304 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:44.990489 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:44.990528 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:44.995745 29685 instancegroups.go:503] Cluster validated. I0724 17:16:44.995788 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:44.995802 29685 instancegroups.go:503] Cluster validated. I0724 17:16:44.995810 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:44.998580 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.001751 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.001792 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.003020 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.003055 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.008267 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.008326 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.009201 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.009242 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.010412 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.010458 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.015704 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.015747 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.015768 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.015782 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.016836 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.017728 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.017781 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.018945 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.018987 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.024295 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.024350 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.025236 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.025273 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.026443 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.026486 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.031803 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.031854 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.032696 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.032736 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.033893 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.033938 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.039251 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.039282 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.039305 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.039319 29685 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0724 17:16:45.040351 29685 instancegroups.go:400] Draining the node: "node-2a.local". I0724 17:16:45.041133 29685 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0724 17:16:45.041171 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.042347 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.042390 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.047666 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.047715 29685 instancegroups.go:400] Draining the node: "node-2b.local". I0724 17:16:45.048660 29685 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0724 17:16:45.048699 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.049876 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.049924 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.055224 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.055274 29685 instancegroups.go:400] Draining the node: "node-2c.local". I0724 17:16:45.056110 29685 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0724 17:16:45.056148 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.057305 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.057349 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.062669 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.062703 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0724 17:16:45.063176 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.063209 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.064382 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.064460 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.064528 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.064553 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.064601 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.065821 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.065873 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.065888 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.065916 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.067056 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.067098 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.067169 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.067195 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.067235 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.068449 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.068501 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.068515 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.068543 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.069717 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.069783 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.069806 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.069850 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.071070 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.071099 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.071164 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.071183 29685 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0724 17:16:45.071210 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.072373 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.072439 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.072465 29685 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0724 17:16:45.072507 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.073706 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. W0724 17:16:45.073763 29685 instancegroups.go:397] Not draining cluster nodes as 'cloudonly' flag is set. I0724 17:16:45.073778 29685 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0724 17:16:45.073805 29685 instancegroups.go:436] waiting for 1ms after terminating instance W0724 17:16:45.074978 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0724 17:16:45.075097 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0724 17:16:45.075636 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.075688 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.076852 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.076894 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.076914 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.076996 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.077024 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.077040 29685 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.077054 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.077828 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.078575 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.078614 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.079807 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.079848 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.079869 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.079910 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.080763 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.080802 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.081996 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.082036 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.082058 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.082080 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.082100 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.082141 29685 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.082163 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.083153 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.083999 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.084038 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.085234 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.085274 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.085296 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.085336 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.086257 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.086296 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.087483 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.087524 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.087546 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.087586 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.088596 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.088635 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.089827 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.089868 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.089889 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.089904 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.089923 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.089937 29685 instancegroups.go:479] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.089950 29685 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0724 17:16:45.091094 29685 instancegroups.go:400] Draining the node: "node-2a.local". I0724 17:16:45.091898 29685 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0724 17:16:45.091937 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.093128 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.093168 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.093190 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.093230 29685 instancegroups.go:400] Draining the node: "node-2b.local". I0724 17:16:45.094127 29685 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0724 17:16:45.094167 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.095371 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.095412 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.095439 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.095480 29685 instancegroups.go:400] Draining the node: "node-2c.local". I0724 17:16:45.096355 29685 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0724 17:16:45.096393 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.097585 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.097627 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. W0724 17:16:45.097648 29685 instancegroups.go:479] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0724 17:16:45.097667 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0724 17:16:45.097967 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0724 17:16:45.098275 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.098306 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.099499 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.099535 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.104797 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.104889 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.104949 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.104981 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.106291 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.107414 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.107458 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.108654 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.108689 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.113950 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.114028 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.115250 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.115309 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.116520 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.116552 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.121830 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.121868 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.121888 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.121902 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.122925 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.123775 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.123815 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.125019 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.125055 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.130295 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.130398 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.131374 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.131414 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.132622 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.132658 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.137929 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.137979 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.138976 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.139015 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.140217 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.140252 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.145504 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.145537 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.145556 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.145570 29685 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0724 17:16:45.146727 29685 instancegroups.go:400] Draining the node: "node-2a.local". I0724 17:16:45.147622 29685 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0724 17:16:45.147662 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.148866 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.148901 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.154161 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.154212 29685 instancegroups.go:400] Draining the node: "node-2b.local". I0724 17:16:45.155096 29685 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0724 17:16:45.155134 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.156342 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.156377 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.161649 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.161711 29685 instancegroups.go:400] Draining the node: "node-2c.local". I0724 17:16:45.162528 29685 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0724 17:16:45.162567 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.163736 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.163773 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.169052 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.169086 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0724 17:16:45.169223 29685 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0724 17:16:45.169735 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.169770 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.170973 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.171016 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.171060 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0724 17:16:45.171492 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.171538 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.173730 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.173783 29685 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0724 17:16:45.173815 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0724 17:16:45.174080 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.174116 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.174133 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0724 17:16:45.174292 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.174317 29685 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0724 17:16:45.174332 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0724 17:16:45.174559 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.174589 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.175757 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.175808 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.181134 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.181200 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.181224 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.181241 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.181965 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.182798 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.182837 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.183993 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.184054 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.184088 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0724 17:16:45.184500 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.184542 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.185755 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.185795 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.191058 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.191147 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.191198 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.191249 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.192153 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.193185 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.193236 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.194430 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.194474 29685 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0724 17:16:45.194496 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0724 17:16:45.194804 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.194831 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.194844 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.196132 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.198331 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.198375 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.199566 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.199602 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.204807 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.204857 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.205713 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.205753 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.206943 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.206991 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.212247 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.212298 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.213121 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.213159 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.214355 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.214392 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.219631 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.219665 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.03s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0724 17:16:45.219964 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.219995 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.220013 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0724 17:16:45.220247 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.220275 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.220291 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0724 17:16:45.220558 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.220586 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.220601 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0724 17:16:45.220762 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.220791 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.220804 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.221833 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.222611 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.222649 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.223835 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.223882 29685 instancegroups.go:523] Cluster did not pass validation within deadline: testing failure. E0724 17:16:45.223904 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0724 17:16:45.224131 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.224164 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.224177 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.225167 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.226001 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.226043 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.227222 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.227265 29685 instancegroups.go:513] Cluster did not validate within deadline: testing validation error. E0724 17:16:45.227288 29685 instancegroups.go:475] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0724 17:16:45.227576 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.227608 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.228804 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.228866 29685 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0724 17:16:45.230038 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.235333 29685 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0724 17:16:45.236514 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.241793 29685 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0724 17:16:45.243006 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.248283 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.248359 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.248385 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.248398 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.249217 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.249989 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.250027 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.251220 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.251264 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.256566 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.256624 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.257589 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.257633 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.258843 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.258928 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.264233 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.264273 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.264301 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.264316 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.265381 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.266180 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.266217 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.267406 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.267457 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.272727 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.272791 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.273619 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.273680 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.274884 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.274935 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.280266 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.280392 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.281863 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.281918 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.283127 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.283183 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.288449 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.288495 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.288545 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.288570 29685 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0724 17:16:45.290385 29685 instancegroups.go:400] Draining the node: "node-2a.local". I0724 17:16:45.292029 29685 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0724 17:16:45.292091 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.293295 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.293352 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.298615 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.298694 29685 instancegroups.go:400] Draining the node: "node-2b.local". I0724 17:16:45.300350 29685 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0724 17:16:45.300411 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.301596 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.301653 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.306915 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.307015 29685 instancegroups.go:400] Draining the node: "node-2c.local". I0724 17:16:45.308705 29685 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0724 17:16:45.308766 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.309972 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.310029 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.315676 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.315719 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I0724 17:16:45.316282 29685 instancegroups.go:593] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0724 17:16:45.316337 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.317545 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.317597 29685 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0724 17:16:45.318758 29685 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0724 17:16:45.319937 29685 instancegroups.go:526] Cluster did not pass validation, will retry in "1ms": testing failure. I0724 17:16:45.321104 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.326347 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.326436 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.326470 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.326498 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.327775 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.329113 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.329168 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.330381 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.330421 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.335622 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.335670 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.336382 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.336414 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.337558 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.337596 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.342861 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.342893 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.342909 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.342920 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.343712 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.344327 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.344355 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.345489 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.345522 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.350767 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.350822 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.351684 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.351724 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.352932 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.352969 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.358217 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.358270 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.359118 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.359157 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.360409 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.360456 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.365719 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.365775 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.365801 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.365816 29685 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0724 17:16:45.366888 29685 instancegroups.go:400] Draining the node: "node-2a.local". I0724 17:16:45.367871 29685 instancegroups.go:591] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0724 17:16:45.367910 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.369082 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.369141 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.374395 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.374456 29685 instancegroups.go:400] Draining the node: "node-2b.local". I0724 17:16:45.375594 29685 instancegroups.go:591] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0724 17:16:45.375636 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.376787 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.376833 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.382088 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.382148 29685 instancegroups.go:400] Draining the node: "node-2c.local". I0724 17:16:45.383146 29685 instancegroups.go:591] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0724 17:16:45.383186 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.384351 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.384387 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.389642 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.389677 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0724 17:16:45.390509 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.390537 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.390550 29685 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0724 17:16:45.391482 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.392476 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.392513 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.393716 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.393752 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.399011 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.399061 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.400073 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.400112 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.401319 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.401355 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.406589 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.406623 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0724 17:16:45.406940 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.406989 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.407014 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.407857 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.408804 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.408842 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.410046 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.410081 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.415332 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.415381 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.416367 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.416414 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.417616 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.417659 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.422945 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.422978 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0724 17:16:45.423348 29685 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled W0724 17:16:45.423390 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0724 17:16:45.423411 29685 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled W0724 17:16:45.423425 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0724 17:16:45.423444 29685 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled W0724 17:16:45.423455 29685 instancegroups.go:468] Not validating cluster as cloudonly flag is set. I0724 17:16:45.423473 29685 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0724 17:16:45.423488 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0724 17:16:45.423806 29685 instancegroups.go:570] Detaching instance "bastion-1a", in group "bastion-1". I0724 17:16:45.423864 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:45.425061 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.425096 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.430370 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.430410 29685 instancegroups.go:185] Rolling updates for InstanceGroup bastion-1 are disabled I0724 17:16:45.430481 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.430511 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.430528 29685 instancegroups.go:311] Tainting 2 nodes in "master-1" instancegroup. I0724 17:16:45.431411 29685 instancegroups.go:185] Rolling updates for InstanceGroup master-1 are disabled I0724 17:16:45.431444 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.431464 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.431476 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:45.432593 29685 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0724 17:16:45.432653 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:45.433837 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.433874 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.439115 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.439153 29685 instancegroups.go:185] Rolling updates for InstanceGroup node-1 are disabled I0724 17:16:45.439168 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.439203 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.439219 29685 instancegroups.go:311] Tainting 3 nodes in "node-2" instancegroup. I0724 17:16:45.440756 29685 instancegroups.go:568] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0724 17:16:45.440818 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:45.442011 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.442047 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:45.447283 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.447320 29685 instancegroups.go:185] Rolling updates for InstanceGroup node-2 are disabled I0724 17:16:45.447338 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0724 17:16:45.447674 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.447711 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.447737 29685 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0724 17:16:45.452705 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.453782 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.453833 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.455028 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.455078 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.455114 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.455169 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.455811 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.456268 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.456300 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.457498 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.457663 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.558369 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.558480 29685 instancegroups.go:400] Draining the node: "node-1d.local". I0724 17:16:45.558507 29685 instancegroups.go:400] Draining the node: "node-1e.local". I0724 17:16:45.560067 29685 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0724 17:16:45.560403 29685 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0724 17:16:45.560450 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.561629 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.561680 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.561724 29685 instancegroups.go:400] Draining the node: "node-1f.local". I0724 17:16:45.562726 29685 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0724 17:16:45.562775 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.563940 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.564014 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.664991 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.665083 29685 instancegroups.go:400] Draining the node: "node-1g.local". I0724 17:16:45.666612 29685 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0724 17:16:45.666687 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.667899 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.667948 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.667966 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0724 17:16:45.668311 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.668349 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.668364 29685 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0724 17:16:45.670550 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:45.670556 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:45.671935 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:45.672218 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:45.672261 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.673466 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.673536 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.774082 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.774151 29685 instancegroups.go:400] Draining the node: "node-1d.local". I0724 17:16:45.774220 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:45.775588 29685 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0724 17:16:45.775878 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:45.775922 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.777109 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.777155 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.777195 29685 instancegroups.go:400] Draining the node: "node-1e.local". I0724 17:16:45.778061 29685 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0724 17:16:45.778108 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.779303 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.779373 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.879984 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.880066 29685 instancegroups.go:400] Draining the node: "node-1f.local". I0724 17:16:45.880983 29685 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0724 17:16:45.881047 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.882228 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.882272 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.882301 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0724 17:16:45.882584 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.882619 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.882634 29685 instancegroups.go:311] Tainting 7 nodes in "master-1" instancegroup. I0724 17:16:45.884873 29685 instancegroups.go:400] Draining the node: "master-1a.local". I0724 17:16:45.885676 29685 instancegroups.go:591] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0724 17:16:45.885724 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.886889 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.886945 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.887032 29685 instancegroups.go:400] Draining the node: "master-1c.local". I0724 17:16:45.887068 29685 instancegroups.go:400] Draining the node: "master-1b.local". I0724 17:16:45.888523 29685 instancegroups.go:591] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0724 17:16:45.889001 29685 instancegroups.go:591] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0724 17:16:45.889074 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.890270 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.890370 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.990919 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.991016 29685 instancegroups.go:400] Draining the node: "master-1e.local". I0724 17:16:45.991081 29685 instancegroups.go:400] Draining the node: "master-1d.local". I0724 17:16:45.992550 29685 instancegroups.go:591] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0724 17:16:45.992918 29685 instancegroups.go:591] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0724 17:16:45.992972 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.994151 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.994215 29685 instancegroups.go:503] Cluster validated. I0724 17:16:45.994274 29685 instancegroups.go:400] Draining the node: "master-1f.local". I0724 17:16:45.995587 29685 instancegroups.go:591] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0724 17:16:45.995652 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:45.996861 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:45.996932 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.097502 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.097584 29685 instancegroups.go:400] Draining the node: "master-1g.local". I0724 17:16:46.098542 29685 instancegroups.go:591] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0724 17:16:46.098597 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.099799 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.099849 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.099869 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0724 17:16:46.100149 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.100187 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.100202 29685 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0724 17:16:46.102131 29685 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0724 17:16:46.102205 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.103403 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.103450 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.103465 29685 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0724 17:16:46.103523 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.104673 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.104717 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.104762 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.104835 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.106101 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.106407 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.106452 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.107644 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.107718 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.208209 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.208290 29685 instancegroups.go:400] Draining the node: "node-1d.local". I0724 17:16:46.208360 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:46.209765 29685 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0724 17:16:46.210077 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:46.210124 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.211334 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.211385 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.211427 29685 instancegroups.go:400] Draining the node: "node-1e.local". I0724 17:16:46.212269 29685 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0724 17:16:46.212321 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.213528 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.213592 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.314112 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.314178 29685 instancegroups.go:400] Draining the node: "node-1f.local". I0724 17:16:46.315040 29685 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0724 17:16:46.315094 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.316300 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.316348 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.316367 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0724 17:16:46.316629 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.316666 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.316680 29685 instancegroups.go:311] Tainting 6 nodes in "node-1" instancegroup. I0724 17:16:46.318491 29685 instancegroups.go:568] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0724 17:16:46.318542 29685 instancegroups.go:568] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0724 17:16:46.318588 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.319783 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.319829 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.319873 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.319947 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.321220 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.321504 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.321549 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.322754 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.322827 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.423405 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.423472 29685 instancegroups.go:400] Draining the node: "node-1d.local". I0724 17:16:46.423543 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:46.424907 29685 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0724 17:16:46.425301 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:46.425359 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.426578 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.426624 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.426667 29685 instancegroups.go:400] Draining the node: "node-1e.local". I0724 17:16:46.427684 29685 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0724 17:16:46.427737 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.428913 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.429005 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.529858 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.529945 29685 instancegroups.go:400] Draining the node: "node-1f.local". I0724 17:16:46.531175 29685 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0724 17:16:46.531238 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.532448 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.532498 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.532516 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0724 17:16:46.532835 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.532864 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.532886 29685 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0724 17:16:46.533743 29685 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0724 17:16:46.533780 29685 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0724 17:16:46.533818 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.535013 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.535048 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.540328 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.540390 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.540463 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.541895 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.541936 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.542256 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.542292 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.543144 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.543179 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.548499 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.548534 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0724 17:16:46.548759 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.548788 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.548801 29685 instancegroups.go:311] Tainting 2 nodes in "node-1" instancegroup. I0724 17:16:46.549598 29685 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0724 17:16:46.549660 29685 instancegroups.go:160] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0724 17:16:46.549673 29685 instancegroups.go:568] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0724 17:16:46.549712 29685 instancegroups.go:160] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0724 17:16:46.549746 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.550945 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.550981 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.556230 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.556312 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.556350 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.557823 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.557866 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.558271 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.558323 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.559032 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.559077 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.564409 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.564444 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0724 17:16:46.564673 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.564708 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.564721 29685 instancegroups.go:311] Tainting 4 nodes in "node-1" instancegroup. I0724 17:16:46.566461 29685 instancegroups.go:568] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0724 17:16:46.566525 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.567694 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.567740 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.567754 29685 instancegroups.go:568] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0724 17:16:46.567809 29685 instancegroups.go:172] waiting for 1ms after detaching instance I0724 17:16:46.569007 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.569050 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.569100 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:46.569174 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.569228 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.571058 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:46.571114 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.571522 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.571592 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.571909 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.571962 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.572318 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.672909 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.672988 29685 instancegroups.go:400] Draining the node: "node-1d.local". I0724 17:16:46.673936 29685 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0724 17:16:46.673990 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.675129 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.675176 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.675194 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0724 17:16:46.675465 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.675504 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.675518 29685 instancegroups.go:311] Tainting 7 nodes in "node-1" instancegroup. I0724 17:16:46.677802 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.678572 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.678619 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.679816 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.679862 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.679906 29685 instancegroups.go:400] Draining the node: "node-1e.local". I0724 17:16:46.679978 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:46.681308 29685 instancegroups.go:591] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0724 17:16:46.681588 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:46.681633 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.682830 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.682903 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.783853 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.783940 29685 instancegroups.go:400] Draining the node: "node-1g.local". I0724 17:16:46.784010 29685 instancegroups.go:400] Draining the node: "node-1f.local". I0724 17:16:46.785353 29685 instancegroups.go:591] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0724 17:16:46.785611 29685 instancegroups.go:591] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0724 17:16:46.785663 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.786840 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.786905 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.786969 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.788433 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.788505 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.789715 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.789786 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.890321 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.890403 29685 instancegroups.go:400] Draining the node: "node-1d.local". I0724 17:16:46.891270 29685 instancegroups.go:591] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0724 17:16:46.891318 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.892542 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.892594 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.892613 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0724 17:16:46.892853 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.892880 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.892900 29685 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0724 17:16:46.892924 29685 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0724 17:16:46.892943 29685 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0724 17:16:46.892959 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0724 17:16:46.893141 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.893169 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.893182 29685 instancegroups.go:311] Tainting 3 nodes in "node-1" instancegroup. I0724 17:16:46.894196 29685 instancegroups.go:132] deleting warm pool instance "node-1-wp-a" I0724 17:16:46.894235 29685 instancegroups.go:132] deleting warm pool instance "node-1-wp-b" I0724 17:16:46.894255 29685 instancegroups.go:132] deleting warm pool instance "node-1-wp-c" I0724 17:16:46.894298 29685 instancegroups.go:400] Draining the node: "node-1a.local". I0724 17:16:46.895051 29685 instancegroups.go:591] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0724 17:16:46.895089 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.896286 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.896328 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.901587 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.901638 29685 instancegroups.go:400] Draining the node: "node-1b.local". I0724 17:16:46.902468 29685 instancegroups.go:591] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0724 17:16:46.902507 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.903714 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.903750 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.908999 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.909049 29685 instancegroups.go:400] Draining the node: "node-1c.local". I0724 17:16:46.909914 29685 instancegroups.go:591] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0724 17:16:46.909954 29685 instancegroups.go:436] waiting for 1ms after terminating instance I0724 17:16:46.911146 29685 instancegroups.go:470] Validating the cluster. I0724 17:16:46.911181 29685 instancegroups.go:506] Cluster validated; revalidating in 5ms to make sure it does not flap. I0724 17:16:46.916429 29685 instancegroups.go:503] Cluster validated. I0724 17:16:46.916462 29685 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 2.017s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.084s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.013s === RUN TestParse E0724 17:15:48.729540 15971 versions.go:96] unable to parse Kubernetes version "" E0724 17:15:48.729626 15971 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.045s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.027s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.10s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.161s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.103s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.081s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.067s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0724 17:16:18.302035 27465 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.068s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I0724 17:16:26.175475 28883 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/pollinterval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0724 17:16:26.220920 28883 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0724 17:16:26.221092 28883 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0724 17:16:26.222200 28883 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0724 17:16:26.222301 28883 model.go:480] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/pollinterval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.121s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I0724 17:16:25.441176 28734 featureflag.go:162] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.089s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.102s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.064s === RUN TestServerGroupBuilder I0724 17:16:26.646461 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.669647 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.683215 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.702899 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.758179 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.772547 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.789577 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.801608 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.815514 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.871414 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.880610 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.889846 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.897248 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:26.903261 29026 featureflag.go:162] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.26s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.357s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.029s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.007s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.059s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.00s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.23s) --- PASS: TestIssueCert/ca (0.40s) --- PASS: TestIssueCert/client (0.39s) --- PASS: TestIssueCert/clientOneYear (0.43s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.237s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.063s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I0724 17:16:23.049208 28080 routetable.go:53] DescribeRouteTables: { } I0724 17:16:23.049305 28080 aws.go:225] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W0724 17:16:23.049325 28080 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0724 17:16:23.049403 28080 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0724 17:16:23.049480 28080 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0724 17:16:23.049572 28080 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0724 17:16:23.049655 28080 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0724 17:16:23.049684 28080 volumes.go:128] DescribeVolumes: { } W0724 17:16:23.049724 28080 aws.go:2239] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0724 17:16:23.049833 28080 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:23.050039 28080 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:23.050399 28080 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:23.050457 28080 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:23.050517 28080 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:23.050565 28080 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:23.050583 28080 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:23.050620 28080 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:23.050643 28080 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:23.050663 28080 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0724 17:16:23.050699 28080 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0724 17:16:23.050725 28080 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0724 17:16:23.050753 28080 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0724 17:16:23.050772 28080 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0724 17:16:23.050798 28080 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0724 17:16:23.050828 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0724 17:16:23.050851 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0724 17:16:23.050874 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0724 17:16:23.050894 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0724 17:16:23.050916 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0724 17:16:23.050935 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0724 17:16:23.050959 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0724 17:16:23.050979 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0724 17:16:23.051002 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0724 17:16:23.051022 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0724 17:16:23.051053 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0724 17:16:23.051075 28080 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.073s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.034s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.085s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.035s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.034s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.066s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.007s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.014s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.01s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.01s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.100s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.070s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.004s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0724 17:15:28.350719 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350796 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350811 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350827 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350841 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350856 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350869 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350883 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350896 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350909 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350922 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350938 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350951 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350969 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.350982 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.350999 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.353479 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.353509 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.353596 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.353614 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.353733 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.353756 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.353842 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.353861 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.353939 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.353958 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354057 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354102 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354187 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354207 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354290 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354310 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354388 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354407 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354484 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354503 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354597 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354617 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354698 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354717 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354804 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354823 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354901 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.354920 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.354998 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.355017 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.355722 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.355747 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.355829 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.355849 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.355925 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.355943 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356020 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356038 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356148 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356169 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356249 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356267 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356350 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356368 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356444 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356462 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356542 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356561 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356637 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356683 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356769 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356788 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356868 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356886 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.356962 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.356990 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357069 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357088 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357164 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357187 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357274 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357304 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357390 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357411 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357489 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357508 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357583 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357602 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357677 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357695 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357787 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357808 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.357906 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.357926 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358003 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358025 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358127 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358145 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358236 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358254 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358333 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358350 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358454 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358472 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358552 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358577 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358654 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358671 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358743 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358761 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358839 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358856 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.358934 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.358951 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359029 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359046 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359120 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359137 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359214 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359231 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359309 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359327 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359402 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359419 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359496 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359513 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359600 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359617 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359694 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359711 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359785 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359803 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359880 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359897 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.359974 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.359991 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360067 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360085 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360160 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360177 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360252 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360269 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360353 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360371 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360452 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360470 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360579 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360599 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360676 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360693 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360767 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360784 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360858 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360876 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.360973 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.360991 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361072 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361091 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361166 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361186 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361262 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361279 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361354 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361372 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361453 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361470 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361548 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361565 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361647 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361694 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361769 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361785 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361860 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361877 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.361960 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.361979 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362091 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362109 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362189 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362206 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362285 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362302 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362379 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362396 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362473 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362492 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362573 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362591 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362695 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362713 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362795 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362815 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362892 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.362909 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.362988 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363005 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363083 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363102 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363199 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363217 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363293 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363310 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363387 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363403 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363478 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363495 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363573 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363590 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363697 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363717 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363801 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363827 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.363905 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.363924 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.364002 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.364021 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.364098 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.364116 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0724 17:15:28.364913 5808 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0724 17:15:28.364931 5808 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0724 17:15:28.364945 5808 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0724 17:15:28.364959 5808 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0724 17:15:28.364972 5808 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0724 17:15:28.364986 5808 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0724 17:15:28.364998 5808 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0724 17:15:28.365033 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365050 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365064 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.365078 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365092 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.365105 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365119 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.365135 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365150 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.365165 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365179 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365193 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365212 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365227 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.365240 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.365255 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.373803 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.373848 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.373947 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.373967 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374052 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374070 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374154 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374174 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374260 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374278 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374360 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374378 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374494 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374513 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374608 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374626 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374706 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374724 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374807 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374826 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.374911 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.374930 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375017 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375034 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375115 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375134 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375228 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375247 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375330 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375349 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375446 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375465 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375548 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375566 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375641 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375658 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375735 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375757 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375835 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375852 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.375928 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.375946 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376027 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376048 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376164 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376184 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376272 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376290 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376367 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376384 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376464 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376519 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376658 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376684 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376764 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376782 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.376861 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.376880 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.377074 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.377178 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.377421 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.377450 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.377537 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.377556 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.377644 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.377663 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.377850 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.377933 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378028 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378046 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378121 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378138 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378297 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378361 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378461 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378480 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378566 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378585 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378671 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378689 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378769 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378786 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.378883 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.378901 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379007 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379026 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379109 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379126 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379204 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379223 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379307 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379326 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379415 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379434 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379517 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379536 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379615 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379634 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379710 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379729 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379810 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379828 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.379914 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.379932 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380012 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380033 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380113 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380134 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380241 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380259 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380358 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380377 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380455 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380472 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380549 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380567 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380648 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380668 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380753 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380773 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380856 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380875 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.380952 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.380970 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381057 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381076 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381159 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381179 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381280 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381300 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381380 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381398 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381478 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381497 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381578 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381595 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381680 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381698 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.381775 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.381796 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.382268 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.382323 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.382413 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.382475 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.382608 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.382660 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.382776 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.382830 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.382943 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.383015 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.383145 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.383216 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.383376 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.383431 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.383551 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.383602 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.383772 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.383824 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.383944 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.383992 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.384100 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.384149 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.384261 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.384449 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.384568 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.384618 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.384733 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.384786 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.384902 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.384952 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.385069 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.385122 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.385231 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.385281 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.385387 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.385435 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.385547 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.385598 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.385706 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.385757 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.385869 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.385916 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386023 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386071 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386191 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386241 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386353 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386401 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386514 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386564 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386676 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386723 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386833 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386858 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.386939 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.386963 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0724 17:15:28.387054 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0724 17:15:28.387077 5808 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.055s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.038s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.04s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.151s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.034s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0724 17:16:05.781776 23964 files.go:57] Writing file "/tmp/TestWriteFile1780750574/001/SubDir/test1.tmp" I0724 17:16:05.781982 23964 files.go:113] Changing file mode for "/tmp/TestWriteFile1780750574/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.032s === RUN TestBootstrapChannelBuilder_BuildTasks I0724 17:16:32.873386 29476 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:32.873730 29476 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:32.873800 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:32.873858 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:32.873933 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:32.873994 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:32.874028 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:32.874083 29476 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:32.874133 29476 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:32.883876 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:32.934119 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.035868 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.109102 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.166834 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.229453 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.291621 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.351799 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.437442 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.806603 29476 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (0.98s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0724 17:16:33.853522 29476 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:33.853714 29476 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:33.853768 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:33.853799 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:33.853832 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:33.853859 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:33.853876 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:33.853898 29476 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:33.853916 29476 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:33.853934 29476 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0724 17:16:33.855083 29476 zones.go:93] ListHostedZonesPages { } I0724 17:16:33.911636 29476 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.06s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0724 17:16:33.911825 29476 featureflag.go:162] FeatureFlag "ImageDigest"=false I0724 17:16:33.911990 29476 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:33.912041 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0724 17:16:33.912071 29476 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0724 17:16:33.912105 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0724 17:16:33.912132 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0724 17:16:33.912148 29476 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0724 17:16:33.912170 29476 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0724 17:16:33.912187 29476 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0724 17:16:33.913329 29476 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.08s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0724 17:16:34.139788 29476 new_cluster.go:980] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_Image_Required I0724 17:16:34.171324 29476 populate_instancegroup_spec.go:323] Cannot set default Image for CloudProvider="aws" --- PASS: TestPopulateInstanceGroup_Image_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0724 17:16:34.172003 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172025 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172036 29476 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0724 17:16:34.172054 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0724 17:16:34.172065 29476 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0724 17:16:34.172084 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0724 17:16:34.172103 29476 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0724 17:16:34.172123 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172136 29476 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0724 17:16:34.172155 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172167 29476 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0724 17:16:34.172186 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172211 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172231 29476 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0724 17:16:34.172242 29476 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0724 17:16:34.172251 29476 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0724 17:16:34.172261 29476 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0724 17:16:34.172270 29476 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0724 17:16:34.172281 29476 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0724 17:16:34.172291 29476 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0724 17:16:34.172303 29476 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0724 17:16:34.173304 29476 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.444s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0724 17:16:21.937467 27896 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1386355029/001 I0724 17:16:21.940800 27896 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1386355029/001 --- PASS: TestAutoscalingGroupTerraformRender (0.87s) === RUN TestAutoscalingGroupCloudformationRender I0724 17:16:22.808618 27896 target.go:168] Cloudformation output is in /tmp/TestAutoscalingGroupCloudformationRender1967248289/001 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0724 17:16:22.809556 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0724 17:16:22.809676 27896 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0724 17:16:22.809765 27896 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0724 17:16:22.810060 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:22.810208 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:22.810301 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:22.810351 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:22.810634 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:22.810682 27896 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0724 17:16:22.810737 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:22.810851 27896 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0724 17:16:22.810966 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0724 17:16:22.811165 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:22.811226 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:22.811281 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:22.811303 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:22.811341 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:22.811386 27896 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0724 17:16:22.811459 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0724 17:16:22.811870 27896 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0724 17:16:22.811938 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:22.812086 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0724 17:16:22.812145 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:22.812177 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:22.812203 27896 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0724 17:16:22.812288 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:22.812572 27896 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0724 17:16:22.812651 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:22.812683 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:22.812711 27896 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0724 17:16:22.812789 27896 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:22.812989 27896 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0724 17:16:22.813055 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:22.813087 27896 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0724 17:16:22.813135 27896 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0724 17:16:22.813362 27896 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0724 17:16:22.813416 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:22.813474 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:22.813495 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:22.813528 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:22.813553 27896 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0724 17:16:22.813597 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:22.813688 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:22.813716 27896 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0724 17:16:22.813759 27896 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:22.813820 27896 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0724 17:16:22.813862 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0724 17:16:22.813904 27896 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0724 17:16:22.814119 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0724 17:16:22.814174 27896 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0724 17:16:22.814380 27896 internetgateways.go:66] CreateInternetGateway: { } I0724 17:16:22.814461 27896 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0724 17:16:22.814658 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:22.814739 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:22.814818 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:22.815185 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:22.815312 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:22.815375 27896 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0724 17:16:22.815442 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:22.815544 27896 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0724 17:16:22.815662 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0724 17:16:22.815892 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:22.815969 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:22.816046 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:22.816091 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:22.816154 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:22.816225 27896 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0724 17:16:22.816337 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0724 17:16:22.817499 27896 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender2031217783/001 I0724 17:16:22.817838 27896 target.go:168] Cloudformation output is in /tmp/TestLaunchTemplateCloudformationRender2031217783/001 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0724 17:16:22.824812 27896 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender998059925/001 I0724 17:16:22.826477 27896 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender998059925/001 --- PASS: TestLaunchTemplateTerraformRender (0.60s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0724 17:16:23.419553 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:23.419677 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.419873 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0724 17:16:23.419946 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:23.419987 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.420028 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:23.420132 27896 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0724 17:16:23.420441 27896 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0724 17:16:23.420520 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0724 17:16:23.420563 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0724 17:16:23.420800 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:23.420864 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.420935 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.420961 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.421003 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.421038 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:23.421107 27896 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0724 17:16:23.421218 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0724 17:16:23.421994 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:23.422061 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.422224 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0724 17:16:23.422296 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:23.422334 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.422365 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:23.422422 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:23.422607 27896 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0724 17:16:23.422672 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:23.422711 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.422745 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0724 17:16:23.422977 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:23.423124 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.423244 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.423299 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.423371 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.423448 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:23.423544 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:23.423692 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.423808 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0724 17:16:23.424457 27896 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0724 17:16:23.424592 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.424786 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0724 17:16:23.424890 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:23.424960 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.425037 27896 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0724 17:16:23.425255 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.425444 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:23.425677 27896 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:16:23.425673 27896 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0724 17:16:23.425844 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:23.425920 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.425992 27896 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0724 17:16:23.426309 27896 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0724 17:16:23.426432 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.426538 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.426593 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.426662 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.426725 27896 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0724 17:16:23.426812 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:23.426937 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.427024 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.427135 27896 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0724 17:16:23.427682 27896 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0724 17:16:23.427753 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.427901 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0724 17:16:23.427971 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:23.428015 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.428054 27896 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0724 17:16:23.428119 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:23.428289 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.428460 27896 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0724 17:16:23.428434 27896 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0724 17:16:23.428549 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:23.428613 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.428648 27896 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0724 17:16:23.428930 27896 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0724 17:16:23.428992 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.429059 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.429089 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.429125 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.429157 27896 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0724 17:16:23.429211 27896 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0724 17:16:23.429362 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.429425 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.429487 27896 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0724 17:16:23.429714 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0724 17:16:23.429816 27896 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0724 17:16:23.430014 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:23.430067 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.430120 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.430142 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.430237 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:23.430280 27896 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0724 17:16:23.430334 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:23.430385 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.430563 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0724 17:16:23.430601 27896 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0724 17:16:23.430638 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0724 17:16:23.430833 27896 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0724 17:16:23.430882 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.430936 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.430959 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.430999 27896 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0724 17:16:23.431045 27896 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0724 17:16:23.431115 27896 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0724 17:16:23.431423 27896 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:23.431475 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.431595 27896 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0724 17:16:23.431645 27896 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0724 17:16:23.431673 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.431698 27896 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:23.431838 27896 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:23.431886 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0724 17:16:23.431942 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.431964 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.431996 27896 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0724 17:16:23.432021 27896 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0724 17:16:23.432248 27896 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0724 17:16:23.432418 27896 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:23.432485 27896 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0724 17:16:23.432552 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0724 17:16:23.432578 27896 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0724 17:16:23.432643 27896 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 1.564s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.120s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.032s === RUN TestDiskRenderAzure I0724 17:16:11.010872 25330 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0724 17:16:11.011151 25330 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0724 17:16:11.011372 25330 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0724 17:16:11.011761 25330 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0724 17:16:11.011987 25330 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0724 17:16:11.012222 25330 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0724 17:16:11.012418 25330 resourcegroup.go:103] Creating a new Resource Group with name: rg I0724 17:16:11.012428 25330 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0724 17:16:11.012599 25330 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0724 17:16:11.013066 25330 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0724 17:16:11.013269 25330 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0724 17:16:11.013482 25330 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0724 17:16:11.013776 25330 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0724 17:16:11.014009 25330 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.030s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.034s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I0724 17:16:20.084974 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.085826 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:20.086071 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.086639 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:20.086836 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.087050 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.03s) === RUN TestServiceAccount I0724 17:16:20.087372 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.087525 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:20.087704 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.087878 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:20.088056 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.088144 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0724 17:16:20.088426 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.088806 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:20.088981 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.089464 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0724 17:16:20.089645 27696 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0724 17:16:20.089829 27696 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.576s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.041s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.028s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.027s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.010s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.023s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0724 17:16:31.046702 29429 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0724 17:16:31.046792 29429 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0724 17:16:31.046811 29429 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0724 17:16:31.046829 29429 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0724 17:16:31.046841 29429 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0724 17:16:31.046860 29429 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0724 17:16:31.046871 29429 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0724 17:16:31.047902 29429 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.061s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.012s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.006s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.003s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.007s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.051s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0724 17:16:02.476669 23053 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.054s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.006s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.004s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.004s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.068s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0724 17:16:20.142792 27734 s3context.go:90] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0724 17:16:20.142856 27734 s3context.go:238] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.070s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 123.1 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.24.0-r0.apk >>> kops: Build complete at Sun, 24 Jul 2022 17:17:23 +0000 elapsed time 0h 3m 54s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20220724.171330) (2/3) Purging kubernetes (1.24.3-r1) (3/3) Purging go (1.18.4-r2) Executing busybox-1.35.0-r20.trigger OK: 333 MiB in 101 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...