>>> kops: Building testing/kops 1.26.3-r1 (using abuild 3.11.20-r0) started Thu, 13 Jul 2023 10:12:22 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go bash (1/5) Installing kubernetes (1.27.3-r1) (2/5) Installing go (1.20.6-r0) (3/5) Installing readline (8.2.1-r2) (4/5) Installing bash (5.2.15-r5) Executing bash-5.2.15-r5.post-install (5/5) Installing .makedepends-kops (20230713.101224) Executing busybox-1.36.1-r3.trigger OK: 548 MiB in 110 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Cleaning up tmpdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 99 31.3M 99 31.2M 0 0 31.3M 0 --:--:-- --:--:-- --:--:-- 31.3M 100 31.3M 100 31.3M 0 0 21.8M 0 0:00:01 0:00:01 --:--:-- 21.8M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.3.tar.gz >>> kops: Checking sha512sums... kops-1.26.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.26.3.tar.gz... go: downloading github.com/cert-manager/cert-manager v1.11.0 go: downloading github.com/spf13/viper v1.15.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading go.uber.org/multierr v1.10.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading k8s.io/klog/v2 v2.90.1 go: downloading github.com/aws/aws-sdk-go v1.44.217 go: downloading k8s.io/apimachinery v0.26.2 go: downloading google.golang.org/api v0.111.0 go: downloading k8s.io/api v0.26.2 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v1.2.0 go: downloading k8s.io/klog v1.0.0 go: downloading k8s.io/client-go v0.26.2 go: downloading k8s.io/cli-runtime v0.26.2 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.1 go: downloading k8s.io/component-base v0.26.2 go: downloading github.com/google/go-containerregistry v0.13.0 go: downloading github.com/google/go-cmp v0.5.9 go: downloading sigs.k8s.io/controller-runtime v0.14.5 go: downloading golang.org/x/crypto v0.7.0 go: downloading helm.sh/helm/v3 v3.11.2 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/stretchr/testify v1.8.2 go: downloading k8s.io/kubectl v0.26.2 go: downloading golang.org/x/oauth2 v0.6.0 go: downloading golang.org/x/sync v0.1.0 go: downloading k8s.io/cloud-provider-aws v1.26.1 go: downloading github.com/digitalocean/godo v1.97.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.3 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/inconshreveable/mousetrap v1.1.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/cloud-provider-gcp/providers v0.26.2 go: downloading github.com/hetznercloud/hcloud-go v1.41.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading google.golang.org/protobuf v1.28.1 go: downloading golang.org/x/net v0.8.0 go: downloading github.com/google/gnostic v0.6.9 go: downloading k8s.io/apiserver v0.26.2 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/prometheus/common v0.39.0 go: downloading k8s.io/mount-utils v0.26.2 go: downloading k8s.io/utils v0.0.0-20230308161112-d77c459e9343 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/imdario/mergo v0.3.13 go: downloading golang.org/x/term v0.6.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading cloud.google.com/go/compute v1.18.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading k8s.io/kubelet v0.26.2 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/Azure/azure-sdk-for-go v68.0.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.12 go: downloading golang.org/x/sys v0.6.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/scaleway/scaleway-sdk-go v1.0.0-beta.14 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/sergi/go-diff v1.3.1 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/Masterminds/sprig/v3 v3.2.3 go: downloading github.com/pkg/errors v0.9.1 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.334.0 go: downloading github.com/google/go-tpm-tools v0.3.10 go: downloading github.com/google/go-tpm v0.3.3 go: downloading golang.org/x/exp v0.0.0-20230307190834-24139beb5833 go: downloading k8s.io/gengo v0.0.0-20230306165830-ab3349d207d4 go: downloading google.golang.org/appengine v1.6.7 go: downloading google.golang.org/grpc v1.53.0 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading golang.org/x/time v0.3.0 go: downloading k8s.io/cloud-provider v0.26.2 go: downloading k8s.io/csi-translation-lib v0.26.0 go: downloading github.com/subosito/gotenv v1.4.2 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/magiconair/properties v1.8.7 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/pelletier/go-toml/v2 v2.0.6 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading github.com/golang/protobuf v1.5.2 go: downloading k8s.io/apiextensions-apiserver v0.26.1 go: downloading sigs.k8s.io/gateway-api v0.6.0 go: downloading k8s.io/kube-openapi v0.0.0-20221207184640-f3cff1453715 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading golang.org/x/text v0.8.0 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4 go: downloading github.com/moby/sys/mountinfo v0.6.2 go: downloading github.com/docker/cli v20.10.21+incompatible go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.2.0 go: downloading github.com/huandu/xstrings v1.4.0 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.3.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading google.golang.org/genproto v0.0.0-20230223222841-637eb2293923 go: downloading go.opencensus.io v0.24.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/mattn/go-isatty v0.0.17 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/google/go-sev-guest v0.4.1 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/containerd/containerd v1.6.15 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.2 go: downloading github.com/onsi/ginkgo/v2 v2.6.1 go: downloading github.com/onsi/gomega v1.24.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.24.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/distribution/distribution/v3 v3.0.0-20221208165359-362910506bc2 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/dnaeon/go-vcr v1.2.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.3 go: downloading github.com/docker/docker v20.10.21+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/emicklei/go-restful/v3 v3.10.1 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/google/logger v1.1.1 go: downloading github.com/pborman/uuid v1.2.0 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/go-openapi/jsonreference v0.20.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/kr/pretty v0.3.1 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading go.uber.org/atomic v1.10.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading golang.org/x/mod v0.8.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.9.0 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/moby/term v0.0.0-20221205130635-1aeaba878587 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/Microsoft/hcsshim v0.9.6 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/creack/pty v1.1.18 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/felixge/httpsnoop v1.0.3 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.4 go: downloading cloud.google.com/go/longrunning v0.3.0 go: downloading cloud.google.com/go v0.107.0 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.26.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.26.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version=1.26.3 -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0713 10:14:53.281388 12705 addon.go:233] addon test wants to update all nodes I0713 10:14:53.283633 12705 addon.go:233] addon test wants to update workers nodes I0713 10:14:53.284028 12705 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0713 10:14:53.284536 12705 addon.go:266] installing PKI for "test" I0713 10:14:53.917309 12705 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.10s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.187s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.098s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kopscontrollerclient [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/model/scalewaymodel [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/nodeidentity/scaleway [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/scaleway [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/scaleway [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] === RUN TestCreateClusterMinimal I0713 10:14:56.006887 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.007066 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.007109 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.007151 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.007189 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.007240 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.007289 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.007315 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.007345 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.007372 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.013991 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.014047 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.058588 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.058808 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.059227 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.074243 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.074321 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.074348 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.074379 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.074404 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.074435 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.074459 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.074475 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.074497 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.074513 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.078442 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.078490 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.102356 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.102514 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.102799 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.110069 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.110143 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.110169 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.110198 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.110222 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.110253 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.110278 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.110294 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.110317 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.110333 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.114313 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.114385 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.229765 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.229951 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.230244 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.235759 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.235827 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.235853 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.235884 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.235917 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.235951 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.235977 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.235996 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.236022 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.236040 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.240827 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.240899 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.336754 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.337482 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.338411 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.343991 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.344069 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.344095 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.344127 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.344152 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.344184 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.344210 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.344228 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.344253 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.344278 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.348335 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.348376 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.377866 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.378038 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.378332 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.384069 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.384139 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.384162 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.384190 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.384213 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.384243 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.384268 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.384287 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.384309 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.384326 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.419713 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.419788 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.480747 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.480919 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.481207 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.486645 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.486714 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.486738 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.486765 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.486790 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.486821 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.486849 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.486866 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.486890 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.486909 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.490665 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.490702 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.545577 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.545765 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.546063 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.551450 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.551515 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.551538 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.551566 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.551589 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.551618 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.551644 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.551660 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.551683 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.551699 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.555495 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.555546 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.633481 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.633601 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.633909 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:56.639337 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.639417 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.639451 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.639499 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.639535 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.639586 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.639621 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.639652 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.639689 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.639714 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.643581 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.643626 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.708469 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:56.708641 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:56.708971 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.71s) === RUN TestCreateClusterHetzner I0713 10:14:56.715031 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.715144 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.715170 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.715202 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.715227 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.715257 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.715285 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.715302 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.715325 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.715341 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.719679 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.719731 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0713 10:14:56.752669 13927 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0713 10:14:56.752712 13927 new_cluster.go:1354] Cloud Provider ID: "hetzner" I0713 10:14:56.761764 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.761875 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.761911 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.761952 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.761989 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.762034 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.762082 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.762114 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.762156 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.762184 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.766904 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.766994 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.846254 13927 new_cluster.go:1354] Cloud Provider ID: "hetzner" --- PASS: TestCreateClusterHetzner (0.14s) === RUN TestCreateClusterOpenStack I0713 10:14:56.854272 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.854359 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.854386 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.854422 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.854446 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.854477 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.854504 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.854520 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.854544 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.854562 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.858424 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.858470 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0713 10:14:56.879499 13927 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0713 10:14:56.879551 13927 new_cluster.go:1354] Cloud Provider ID: "openstack" I0713 10:14:56.882176 13927 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 --- PASS: TestCreateClusterOpenStack (0.04s) === RUN TestCreateClusterOpenStackOctavia I0713 10:14:56.890635 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.890725 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.890753 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.890784 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.890810 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.890843 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.890869 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.890887 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.890910 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.890925 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.894859 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.894905 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0713 10:14:56.922451 13927 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0713 10:14:56.922493 13927 new_cluster.go:1354] Cloud Provider ID: "openstack" I0713 10:14:56.925097 13927 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0713 10:14:56.925135 13927 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackOctavia (0.05s) === RUN TestCreateClusterOpenStackNoDNS I0713 10:14:56.938528 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.938655 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.938691 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.938738 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.938778 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.938828 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.938870 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.938899 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.938939 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.938966 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.942668 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.942707 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0713 10:14:56.982716 13927 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0713 10:14:56.982768 13927 new_cluster.go:1354] Cloud Provider ID: "openstack" I0713 10:14:56.982836 13927 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0713 10:14:56.982857 13927 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackNoDNS (0.06s) === RUN TestCreateClusterCilium I0713 10:14:56.993610 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:56.993704 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:56.993735 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:56.993770 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:56.993795 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:56.993828 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:56.993857 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:56.993876 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:56.993904 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:56.993923 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:56.997754 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:56.997796 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.030245 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.030428 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.030799 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterCilium (0.08s) === RUN TestCreateClusterOverride I0713 10:14:57.069322 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.069453 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.069490 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.069524 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.069551 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.069584 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.069616 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.069634 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.069658 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.069676 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.073510 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.073565 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.200494 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.200991 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.201278 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.14s) === RUN TestCreateClusterKubernetesFeatureGates I0713 10:14:57.206471 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.206559 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.206583 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.206611 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.206634 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.206664 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.206688 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.206703 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.206725 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.206744 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.210530 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.210580 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.273186 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.273362 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.273734 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterKubernetesFeatureGates (0.07s) === RUN TestCreateClusterComplex I0713 10:14:57.280212 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.280294 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.280319 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.280350 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.280375 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.280405 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.280433 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.280449 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.280471 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.280488 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.284388 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.284446 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.356568 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.356751 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.357042 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.08s) === RUN TestCreateClusterHA I0713 10:14:57.362683 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.362790 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.362824 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.362865 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.362897 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.362937 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.362974 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.363005 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.363039 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.363061 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.367745 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.367808 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.413857 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.414180 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.414196 13927 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0713 10:14:57.414206 13927 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0713 10:14:57.414528 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:57.425514 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.425602 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.425630 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.425685 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.425714 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.425745 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.425772 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.425788 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.425813 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.425831 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.430215 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.430260 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.515414 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.515816 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.515834 13927 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0713 10:14:57.515846 13927 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0713 10:14:57.516278 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.16s) === RUN TestCreateClusterMinimalGCE I0713 10:14:57.524602 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.524700 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.524726 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.524763 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.524789 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.524825 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.524854 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.524880 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.524907 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.524924 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.528642 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.528707 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.550771 13927 new_cluster.go:1354] Cloud Provider ID: "gce" I0713 10:14:57.550890 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 I0713 10:14:57.557480 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.557564 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.557591 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.557620 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.557646 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.557677 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.557703 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.557720 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.557747 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.557764 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.561836 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.561911 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.612865 13927 new_cluster.go:1354] Cloud Provider ID: "gce" I0713 10:14:57.612985 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterMinimalGCE (0.10s) === RUN TestCreateClusterHAGCE I0713 10:14:57.623929 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.624024 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.624051 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.624082 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.624107 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.624138 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.624164 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.624180 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.624204 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.624221 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.627924 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.627965 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.696218 13927 new_cluster.go:1354] Cloud Provider ID: "gce" I0713 10:14:57.696340 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.08s) === RUN TestCreateClusterGCE I0713 10:14:57.704948 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.705043 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.705069 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.705100 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.705128 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.705158 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.705185 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.705204 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.705229 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.705248 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.709040 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.709077 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.748369 13927 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0713 10:14:57.748450 13927 new_cluster.go:1354] Cloud Provider ID: "gce" I0713 10:14:57.748522 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.05s) === RUN TestCreateClusterHASharedZone I0713 10:14:57.754081 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.754173 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.754199 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.754229 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.754253 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.754283 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.754310 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.754329 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.754353 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.754369 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.758161 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.758202 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0713 10:14:57.820929 13927 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0713 10:14:57.820974 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.821206 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.821509 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.08s) === RUN TestCreateClusterHASharedZones I0713 10:14:57.831039 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.831146 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.831176 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.831208 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.831233 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.831262 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.831291 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.831309 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.831333 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.831351 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.835150 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.835198 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0713 10:14:57.916757 13927 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0713 10:14:57.916804 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.917185 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.917202 13927 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0713 10:14:57.917560 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.10s) === RUN TestCreateClusterPrivate I0713 10:14:57.926693 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.926788 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.926817 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.926847 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.926879 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.926909 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.926944 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.926964 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.926994 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.927012 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.930877 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.930916 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.967508 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:57.967720 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:57.967737 13927 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0713 10:14:57.968054 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:57.968082 13927 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.05s) === RUN TestCreateClusterPrivateGCE I0713 10:14:57.974515 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:57.974592 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:57.974614 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:57.974639 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:57.974661 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:57.974689 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:57.974713 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:57.974729 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:57.974751 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:57.974768 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:57.978312 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:57.978348 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.063732 13927 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0713 10:14:58.063861 13927 new_cluster.go:1354] Cloud Provider ID: "gce" I0713 10:14:58.063991 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.10s) === RUN TestCreateClusterWithNGWSpecified I0713 10:14:58.071641 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.071745 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.071778 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.071861 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.071896 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.071941 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.071975 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.072004 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.072042 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.072066 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.076241 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.076302 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.129413 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.129642 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:58.129659 13927 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0713 10:14:58.129972 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:58.130004 13927 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.06s) === RUN TestCreateClusterWithINGWSpecified I0713 10:14:58.136388 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.136486 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.136513 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.136542 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.136566 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.136598 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.136624 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.136640 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.136667 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.136684 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.140601 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.140665 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.185020 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.185236 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:58.185253 13927 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0713 10:14:58.185579 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:58.185608 13927 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.06s) === RUN TestCreateClusterSharedVPC I0713 10:14:58.192069 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.192162 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.192189 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.192215 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.192267 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.192302 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.192340 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.192357 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.192382 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.192399 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.196440 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.196495 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.241959 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.242121 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.242172 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.242213 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.242230 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.242269 13927 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0713 10:14:58.242582 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.06s) === RUN TestCreateClusterSharedSubnets I0713 10:14:58.250365 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.250457 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.250483 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.250515 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.250539 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.250570 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.250595 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.250612 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.250636 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.250653 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.254670 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.254733 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.288364 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.288413 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.288493 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.288649 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.288670 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.288693 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.288710 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.289043 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.04s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0713 10:14:58.294375 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.294469 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.294497 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.294526 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.294550 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.294582 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.294607 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.294625 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.294647 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.294663 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.298212 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.298247 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.381075 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:14:58.381148 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.381183 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.381303 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.381541 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.381580 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.381628 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.381657 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.382147 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.10s) === RUN TestCreateClusterPrivateSharedSubnets I0713 10:14:58.391073 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.391215 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.391256 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.391303 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.391345 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.391397 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.391444 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.391480 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.391522 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.391551 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.395733 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.395788 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.454837 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.454886 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.454943 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.454960 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.455017 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.455159 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.455179 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.455201 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:14:58.455216 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:14:58.455553 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.07s) === RUN TestCreateClusterIPv6 I0713 10:14:58.461062 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.461152 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.461177 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.461202 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.461224 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.461257 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.461285 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.461304 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.461329 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.461346 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.465528 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.465576 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.493886 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.494062 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet dualstack-us-test-1a I0713 10:14:58.494078 13927 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0713 10:14:58.494603 13927 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.04s) === RUN TestCreateClusterDifferentAMIs I0713 10:14:58.503302 13927 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0713 10:14:58.503321 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.503402 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.503428 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.503458 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.503484 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.503515 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.503541 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.503558 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.503582 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.503598 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.507396 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.507438 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.527293 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.527399 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:58.527428 13927 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0713 10:14:58.527787 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:58.535655 13927 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.03s) === RUN TestCreateClusterKarpenter I0713 10:14:58.535778 13927 featureflag.go:166] FeatureFlag "Karpenter"=true I0713 10:14:58.535792 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.535904 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.535934 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.535966 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.535992 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.536028 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.536060 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.536079 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.536109 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.536129 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.540305 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.540363 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0713 10:14:58.617713 13927 new_cluster.go:1354] Cloud Provider ID: "aws" I0713 10:14:58.617888 13927 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0713 10:14:58.617903 13927 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0713 10:14:58.617914 13927 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0713 10:14:58.618261 13927 zones.go:93] ListHostedZonesPages { } I0713 10:14:58.624345 13927 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.09s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0713 10:14:58.624677 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.624745 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.624769 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.624820 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.624844 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.624876 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.624901 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.624917 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.624944 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.624961 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.630102 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0713 10:14:58.631282 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:14:58.631356 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:14:58.631375 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:14:58.631399 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:14:58.631422 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:14:58.631447 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:14:58.631501 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:14:58.631520 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:14:58.631543 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:14:58.631555 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:14:58.652545 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:14:58.655483 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:14:58.655580 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0713 10:15:01.333354 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:03.997534 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:03.999994 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:04.000015 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:04.000090 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:04.000422 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:04.005450 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:04.006305 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.006479 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.007446 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.007715 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.007786 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:04.008168 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:04.008390 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:04.010910 13927 target.go:116] Terraform output is in /tmp/test3004555549/out --- PASS: TestMinimal (5.38s) === RUN TestMinimal_v1_23 I0713 10:15:04.012911 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:04.012984 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:04.013009 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:04.013037 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:04.013060 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:04.013093 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:04.013141 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:04.013160 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:04.013182 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:04.013199 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:04.031558 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0713 10:15:04.033416 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.033475 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.478593 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:04.623656 13927 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0713 10:15:04.624379 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:04.624404 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:04.624499 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:04.626410 13927 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0713 10:15:04.629620 13927 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0713 10:15:04.629934 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.630118 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.630857 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.631736 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.631798 13927 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0713 10:15:04.631972 13927 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0713 10:15:04.632142 13927 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0713 10:15:04.634976 13927 target.go:116] Terraform output is in /tmp/test790323720/out --- PASS: TestMinimal_v1_23 (0.62s) === RUN TestMinimal_v1_24 I0713 10:15:04.637213 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:04.637288 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:04.637314 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:04.637340 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:04.637362 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:04.637394 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:04.637445 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:04.637467 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:04.637490 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:04.637506 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:04.659001 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:04.662210 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:04.662391 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.099477 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:05.276014 13927 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0713 10:15:05.276482 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:05.276507 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:05.276593 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:05.278523 13927 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0713 10:15:05.281547 13927 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0713 10:15:05.281929 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.282140 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.283086 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.283652 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.283725 13927 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0713 10:15:05.283912 13927 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0713 10:15:05.284032 13927 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0713 10:15:05.286845 13927 target.go:116] Terraform output is in /tmp/test886156033/out --- PASS: TestMinimal_v1_24 (0.65s) === RUN TestMinimal_v1_25 I0713 10:15:05.289068 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:05.289146 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:05.289171 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:05.289197 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:05.289220 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:05.289252 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:05.289301 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:05.289319 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:05.289341 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:05.289359 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:05.308270 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:05.310203 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.310269 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:05.917285 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:06.065409 13927 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0713 10:15:06.065935 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:06.065972 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:06.066081 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:06.068550 13927 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0713 10:15:06.071868 13927 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0713 10:15:06.072437 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:06.072558 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:06.074039 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:06.074178 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:06.074285 13927 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0713 10:15:06.074541 13927 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0713 10:15:06.074730 13927 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0713 10:15:06.083279 13927 target.go:116] Terraform output is in /tmp/test1346288280/out --- PASS: TestMinimal_v1_25 (0.80s) === RUN TestMinimal_v1_26 I0713 10:15:06.087685 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:06.087852 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:06.087905 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:06.087989 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:06.088058 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:06.088145 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:06.088256 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:06.088313 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:06.088380 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:06.088426 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:06.116322 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:06.119647 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:06.119761 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:06.871461 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.044422 13927 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0713 10:15:07.047250 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:07.047276 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:07.047360 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:07.048560 13927 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0713 10:15:07.051464 13927 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0713 10:15:07.051668 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.052494 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.052971 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.054297 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.054422 13927 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0713 10:15:07.054867 13927 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0713 10:15:07.054965 13927 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0713 10:15:07.057635 13927 target.go:116] Terraform output is in /tmp/test1672947444/out --- PASS: TestMinimal_v1_26 (0.97s) === RUN TestMinimal_NoneDNS I0713 10:15:07.059559 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.059642 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.059674 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.059713 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.059752 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.059795 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.059863 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.059887 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.059909 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.059923 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:07.079855 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.079929 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.295300 13927 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0713 10:15:07.298712 13927 executor.go:111] Tasks: 43 done / 98 total; 20 can run I0713 10:15:07.300791 13927 executor.go:111] Tasks: 63 done / 98 total; 25 can run I0713 10:15:07.301341 13927 executor.go:111] Tasks: 88 done / 98 total; 2 can run I0713 10:15:07.301379 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:07.301440 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:07.301546 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:07.301626 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:07.308635 13927 executor.go:111] Tasks: 90 done / 98 total; 4 can run I0713 10:15:07.308928 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.309074 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.310132 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.311966 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.312165 13927 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0713 10:15:07.312542 13927 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0713 10:15:07.312702 13927 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0713 10:15:07.317081 13927 target.go:116] Terraform output is in /tmp/test2845449666/out --- PASS: TestMinimal_NoneDNS (0.26s) === RUN TestHetzner I0713 10:15:07.321090 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:07.939570 13927 executor.go:111] Tasks: 0 done / 46 total; 37 can run I0713 10:15:07.943844 13927 executor.go:111] Tasks: 37 done / 46 total; 3 can run I0713 10:15:07.944162 13927 executor.go:111] Tasks: 40 done / 46 total; 2 can run I0713 10:15:07.947444 13927 executor.go:111] Tasks: 42 done / 46 total; 4 can run I0713 10:15:07.949994 13927 executor.go:111] Tasks: 46 done / 46 total; 0 can run I0713 10:15:07.951883 13927 target.go:116] Terraform output is in /tmp/test4199455265/out --- PASS: TestHetzner (0.63s) === RUN TestNvidia I0713 10:15:07.954363 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.954479 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.954519 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.954570 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.954610 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.954661 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.954741 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.954769 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.954809 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.954836 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.976788 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:07.978813 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:07.978872 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.056697 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:08.131688 13927 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0713 10:15:08.133133 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:08.133198 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:08.133315 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:08.138004 13927 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0713 10:15:08.142073 13927 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0713 10:15:08.142265 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.142454 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.143397 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.143843 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.143920 13927 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0713 10:15:08.144188 13927 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0713 10:15:08.144346 13927 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0713 10:15:08.147272 13927 target.go:116] Terraform output is in /tmp/test2209632411/out --- PASS: TestNvidia (0.19s) === RUN TestMinimalGossip I0713 10:15:08.149409 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.149504 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.149535 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.149572 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.149600 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.149631 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.149691 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.149709 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.149740 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.149758 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:08.166150 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.166217 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.350592 13927 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0713 10:15:08.353006 13927 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0713 10:15:08.357568 13927 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0713 10:15:08.357912 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.358117 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.358939 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.360109 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.360225 13927 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0713 10:15:08.360477 13927 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0713 10:15:08.360608 13927 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0713 10:15:08.363299 13927 target.go:116] Terraform output is in /tmp/test452215570/out --- PASS: TestMinimalGossip (0.22s) === RUN TestMinimalGossipIRSA I0713 10:15:08.365407 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.365510 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.365543 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.365578 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.365603 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.365634 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.365709 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.365726 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.365752 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.365771 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:08.385857 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.385922 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.562098 13927 executor.go:111] Tasks: 0 done / 89 total; 44 can run I0713 10:15:08.564531 13927 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0713 10:15:08.568102 13927 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0713 10:15:08.568407 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.568623 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.569891 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.570826 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:08.570938 13927 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0713 10:15:08.571202 13927 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0713 10:15:08.571349 13927 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0713 10:15:08.575524 13927 target.go:116] Terraform output is in /tmp/test3796392910/out --- PASS: TestMinimalGossipIRSA (0.21s) === RUN TestMinimalGCE I0713 10:15:08.578888 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.580587 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:09.766010 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:09.766117 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:09.766246 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0713 10:15:09.766286 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0713 10:15:09.766312 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0713 10:15:09.766397 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:09.768089 13927 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0713 10:15:09.772800 13927 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0713 10:15:09.777191 13927 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0713 10:15:09.779742 13927 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0713 10:15:09.780335 13927 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0713 10:15:09.783635 13927 target.go:116] Terraform output is in /tmp/test2302863476/out --- PASS: TestMinimalGCE (1.21s) === RUN TestMinimalGCEPrivate I0713 10:15:09.787036 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.788912 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:10.063623 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:10.063698 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:10.063782 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0713 10:15:10.063802 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0713 10:15:10.063816 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0713 10:15:10.063864 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:10.064809 13927 executor.go:111] Tasks: 0 done / 65 total; 41 can run I0713 10:15:10.071834 13927 executor.go:111] Tasks: 41 done / 65 total; 17 can run I0713 10:15:10.075029 13927 executor.go:111] Tasks: 58 done / 65 total; 5 can run I0713 10:15:10.076674 13927 executor.go:111] Tasks: 63 done / 65 total; 2 can run I0713 10:15:10.076819 13927 executor.go:111] Tasks: 65 done / 65 total; 0 can run I0713 10:15:10.079259 13927 target.go:116] Terraform output is in /tmp/test1339093945/out --- PASS: TestMinimalGCEPrivate (0.29s) === RUN TestMinimalGCEInternalLoadBalancer I0713 10:15:10.081867 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:10.083114 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:10.321907 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:10.321971 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:10.322053 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0713 10:15:10.322075 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0713 10:15:10.322088 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0713 10:15:10.322138 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0713 10:15:10.322169 13927 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0713 10:15:10.323132 13927 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0713 10:15:10.330176 13927 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0713 10:15:10.335628 13927 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0713 10:15:10.335683 13927 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0713 10:15:10.338702 13927 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0713 10:15:10.338840 13927 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0713 10:15:10.339091 13927 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0713 10:15:10.339306 13927 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0713 10:15:10.342388 13927 target.go:116] Terraform output is in /tmp/test3873427027/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.26s) === RUN TestMinimalGCEPublicLoadBalancer I0713 10:15:10.345209 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:10.346989 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:10.588603 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:10.588664 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:10.588781 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/main" W0713 10:15:10.588805 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/events" W0713 10:15:10.588819 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com" W0713 10:15:10.588873 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:10.589975 13927 executor.go:111] Tasks: 0 done / 70 total; 43 can run I0713 10:15:10.592297 13927 executor.go:111] Tasks: 43 done / 70 total; 18 can run I0713 10:15:10.595293 13927 executor.go:111] Tasks: 61 done / 70 total; 7 can run I0713 10:15:10.596926 13927 executor.go:111] Tasks: 68 done / 70 total; 2 can run I0713 10:15:10.597075 13927 executor.go:111] Tasks: 70 done / 70 total; 0 can run I0713 10:15:10.600047 13927 target.go:116] Terraform output is in /tmp/test2186176734/out --- PASS: TestMinimalGCEPublicLoadBalancer (0.26s) === RUN TestMinimalGCELongClusterName I0713 10:15:10.602972 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:10.604690 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:10.853033 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:10.853122 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:10.853218 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0713 10:15:10.853239 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0713 10:15:10.853254 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0713 10:15:10.853309 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:10.854238 13927 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0713 10:15:10.860207 13927 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0713 10:15:10.863721 13927 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0713 10:15:10.865432 13927 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0713 10:15:10.865541 13927 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0713 10:15:10.867583 13927 target.go:116] Terraform output is in /tmp/test2470785265/out --- PASS: TestMinimalGCELongClusterName (0.27s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0713 10:15:10.869611 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:10.870797 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:11.123739 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:11.123803 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:11.123909 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0713 10:15:11.123931 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0713 10:15:11.123947 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0713 10:15:11.124003 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0713 10:15:11.124036 13927 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0713 10:15:11.124953 13927 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0713 10:15:11.127263 13927 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0713 10:15:11.130426 13927 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0713 10:15:11.130462 13927 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0713 10:15:11.132915 13927 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0713 10:15:11.133094 13927 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0713 10:15:11.133204 13927 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0713 10:15:11.133295 13927 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0713 10:15:11.135281 13927 target.go:116] Terraform output is in /tmp/test4122783700/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.27s) === RUN TestMinimalGCEDNSNone I0713 10:15:11.137251 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:11.138745 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:11.373779 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:11.373871 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:11.374018 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0713 10:15:11.374052 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0713 10:15:11.374078 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0713 10:15:11.374163 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0713 10:15:11.374217 13927 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0713 10:15:11.375887 13927 executor.go:111] Tasks: 0 done / 67 total; 41 can run I0713 10:15:11.381797 13927 executor.go:111] Tasks: 41 done / 67 total; 16 can run W0713 10:15:11.386416 13927 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0713 10:15:11.386470 13927 executor.go:111] Tasks: 57 done / 67 total; 5 can run I0713 10:15:11.392285 13927 executor.go:111] Tasks: 62 done / 67 total; 2 can run I0713 10:15:11.392488 13927 executor.go:111] Tasks: 64 done / 67 total; 1 can run I0713 10:15:11.392614 13927 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0713 10:15:11.392795 13927 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0713 10:15:11.394838 13927 target.go:116] Terraform output is in /tmp/test2733420708/out --- PASS: TestMinimalGCEDNSNone (0.26s) === RUN TestHA I0713 10:15:11.396840 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:11.396951 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:11.396984 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:11.397032 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:11.397059 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:11.397105 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:11.397162 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:11.397187 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:11.397221 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:11.397239 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:11.420860 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:11.424142 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.424223 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.424278 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.424328 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.494886 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:11.577150 13927 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0713 10:15:11.577738 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:11.577768 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:11.577851 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:11.579682 13927 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0713 10:15:11.584245 13927 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0713 10:15:11.584518 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.584628 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.584746 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.584978 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.585422 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.586029 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.586405 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.586470 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.586533 13927 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0713 10:15:11.586817 13927 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0713 10:15:11.586943 13927 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0713 10:15:11.591099 13927 target.go:116] Terraform output is in /tmp/test3170287523/out --- PASS: TestHA (0.20s) === RUN TestHighAvailabilityGCE I0713 10:15:11.595780 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:11.598682 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:11.890900 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:11.891013 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:11.891138 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0713 10:15:11.891165 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0713 10:15:11.891186 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0713 10:15:11.891256 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0713 10:15:11.891328 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0713 10:15:11.891391 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:11.893308 13927 executor.go:111] Tasks: 0 done / 82 total; 49 can run I0713 10:15:11.895724 13927 executor.go:111] Tasks: 49 done / 82 total; 19 can run I0713 10:15:11.903216 13927 executor.go:111] Tasks: 68 done / 82 total; 8 can run I0713 10:15:11.905199 13927 executor.go:111] Tasks: 76 done / 82 total; 6 can run I0713 10:15:11.905418 13927 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0713 10:15:11.907992 13927 target.go:116] Terraform output is in /tmp/test2849923660/out --- PASS: TestHighAvailabilityGCE (0.31s) === RUN TestComplex I0713 10:15:11.910518 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:11.910599 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:11.910628 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:11.910687 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:11.910712 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:11.910742 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:11.910772 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:11.910793 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:11.910815 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:11.910834 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:11.936737 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:11.940496 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:11.940579 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.534580 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:12.660886 13927 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0713 10:15:12.661490 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:12.661532 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:12.663661 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:12.666906 13927 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0713 10:15:12.668923 13927 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0713 10:15:12.669443 13927 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0713 10:15:12.669507 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:12.669579 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:12.669675 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:12.669733 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:12.674167 13927 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0713 10:15:12.674356 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.674616 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.676413 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.676785 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.676861 13927 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0713 10:15:12.677125 13927 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0713 10:15:12.677254 13927 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0713 10:15:12.681145 13927 target.go:116] Terraform output is in /tmp/test142201615/out I0713 10:15:12.683418 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:12.683488 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:12.683515 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:12.683545 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:12.683568 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:12.683602 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:12.683628 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:12.683645 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:12.683668 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:12.683686 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:12.703010 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:12.705396 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.705462 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.780578 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:12.940770 13927 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0713 10:15:12.941377 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:12.941415 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:12.941526 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:12.943446 13927 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0713 10:15:12.946139 13927 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0713 10:15:12.946799 13927 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0713 10:15:12.946825 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:12.946885 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:12.946978 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:12.947039 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:12.951219 13927 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0713 10:15:12.951430 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.951677 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.953098 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.954351 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.954465 13927 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0713 10:15:12.954738 13927 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0713 10:15:12.954871 13927 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0713 10:15:12.960075 13927 target.go:116] Terraform output is in /tmp/test2789095805/out --- PASS: TestComplex (1.07s) === RUN TestCompress I0713 10:15:12.975747 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:12.975883 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:12.975929 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:12.975975 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:12.976014 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:12.976061 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:12.976138 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:12.976169 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:12.976205 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:12.976229 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:12.992640 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:12.994434 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:12.994491 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.064692 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:13.151611 13927 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0713 10:15:13.152189 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:13.152227 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:13.152333 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:13.154016 13927 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0713 10:15:13.159007 13927 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0713 10:15:13.159257 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.159419 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.165977 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.166273 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.166355 13927 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0713 10:15:13.166601 13927 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0713 10:15:13.166754 13927 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0713 10:15:13.169380 13927 target.go:116] Terraform output is in /tmp/test327961456/out --- PASS: TestCompress (0.20s) === RUN TestExternalPolicies I0713 10:15:13.171372 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:13.171465 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:13.171496 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:13.171529 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:13.171557 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:13.171589 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:13.171646 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:13.171663 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:13.171687 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:13.171706 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:13.198610 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:13.202120 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.202211 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.273502 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:13.355440 13927 executor.go:111] Tasks: 0 done / 100 total; 45 can run I0713 10:15:13.355936 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:13.356035 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:13.356132 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:13.358893 13927 executor.go:111] Tasks: 45 done / 100 total; 18 can run I0713 10:15:13.362619 13927 executor.go:111] Tasks: 63 done / 100 total; 26 can run I0713 10:15:13.363114 13927 executor.go:111] Tasks: 89 done / 100 total; 3 can run W0713 10:15:13.363223 13927 api.go:53] PageSize not implemented W0713 10:15:13.363324 13927 api.go:53] PageSize not implemented I0713 10:15:13.366011 13927 executor.go:111] Tasks: 92 done / 100 total; 4 can run I0713 10:15:13.366139 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.366354 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.367008 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.369644 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.369766 13927 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0713 10:15:13.370040 13927 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0713 10:15:13.370180 13927 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0713 10:15:13.374223 13927 target.go:116] Terraform output is in /tmp/test2525724708/out --- PASS: TestExternalPolicies (0.21s) === RUN TestMinimalIPv6 I0713 10:15:13.376646 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:13.376726 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:13.376756 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:13.376831 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:13.376870 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:13.376906 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:13.376940 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:13.376960 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:13.376983 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:13.377003 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:13.405272 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:13.407980 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.408068 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.481076 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:13.640599 13927 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0713 10:15:13.642809 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:13.642854 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:13.642951 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:13.646631 13927 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0713 10:15:13.648799 13927 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0713 10:15:13.649345 13927 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0713 10:15:13.649733 13927 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0713 10:15:13.649789 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:13.649862 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:13.649994 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:13.650053 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:13.653137 13927 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0713 10:15:13.653256 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.653329 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.654197 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.655071 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.655139 13927 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0713 10:15:13.655317 13927 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0713 10:15:13.655412 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:13.658964 13927 target.go:116] Terraform output is in /tmp/test1242322937/out --- PASS: TestMinimalIPv6 (0.28s) === RUN TestMinimalIPv6Calico I0713 10:15:13.661276 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:13.661364 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:13.661397 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:13.661466 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:13.661508 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:13.661554 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:13.661590 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:13.661621 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:13.661663 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:13.661688 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:13.679447 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:13.682682 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.682757 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:13.754613 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:14.073137 13927 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0713 10:15:14.073849 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:14.073875 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:14.073959 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:14.075526 13927 executor.go:111] Tasks: 47 done / 126 total; 21 can run I0713 10:15:14.080597 13927 executor.go:111] Tasks: 68 done / 126 total; 33 can run I0713 10:15:14.081164 13927 executor.go:111] Tasks: 101 done / 126 total; 9 can run I0713 10:15:14.081462 13927 executor.go:111] Tasks: 110 done / 126 total; 8 can run I0713 10:15:14.081529 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:14.081611 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:14.081716 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:14.081775 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:14.086190 13927 executor.go:111] Tasks: 118 done / 126 total; 4 can run I0713 10:15:14.086430 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.086598 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.088591 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.088671 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.088748 13927 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0713 10:15:14.089036 13927 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0713 10:15:14.089180 13927 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0713 10:15:14.095724 13927 target.go:116] Terraform output is in /tmp/test2287551275/out --- PASS: TestMinimalIPv6Calico (0.44s) === RUN TestMinimalIPv6Cilium I0713 10:15:14.100657 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:14.100793 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:14.100835 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:14.100928 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:14.100970 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:14.101021 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:14.101065 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:14.101097 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:14.101138 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:14.101168 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:14.134227 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:14.138321 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.138410 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.210462 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:14.378209 13927 executor.go:111] Tasks: 0 done / 125 total; 47 can run I0713 10:15:14.378853 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:14.378881 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:14.378954 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:14.380618 13927 executor.go:111] Tasks: 47 done / 125 total; 21 can run I0713 10:15:14.382946 13927 executor.go:111] Tasks: 68 done / 125 total; 32 can run I0713 10:15:14.383449 13927 executor.go:111] Tasks: 100 done / 125 total; 9 can run I0713 10:15:14.383731 13927 executor.go:111] Tasks: 109 done / 125 total; 8 can run I0713 10:15:14.383769 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:14.383816 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:14.383881 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:14.383956 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:14.389027 13927 executor.go:111] Tasks: 117 done / 125 total; 4 can run I0713 10:15:14.389201 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.389376 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.390156 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.391789 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.391877 13927 executor.go:111] Tasks: 121 done / 125 total; 2 can run I0713 10:15:14.392106 13927 executor.go:111] Tasks: 123 done / 125 total; 2 can run I0713 10:15:14.392239 13927 executor.go:111] Tasks: 125 done / 125 total; 0 can run I0713 10:15:14.395884 13927 target.go:116] Terraform output is in /tmp/test1587187356/out --- PASS: TestMinimalIPv6Cilium (0.30s) === RUN TestMinimalIPv6NoSubnetPrefix I0713 10:15:14.398265 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:14.398343 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:14.398370 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:14.398424 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:14.398448 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:14.398481 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:14.398507 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:14.398524 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:14.398546 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:14.398568 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:14.419370 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:14.421708 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.421772 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.491547 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:14.610708 13927 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0713 10:15:14.611349 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:14.611378 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:14.611469 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:14.613122 13927 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0713 10:15:14.614943 13927 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0713 10:15:14.615512 13927 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0713 10:15:14.615808 13927 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0713 10:15:14.615939 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:14.615994 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:14.616079 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:14.616134 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:14.619277 13927 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0713 10:15:14.619415 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.619615 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.620339 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.621182 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.621248 13927 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0713 10:15:14.621480 13927 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0713 10:15:14.621597 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:14.628793 13927 target.go:116] Terraform output is in /tmp/test4016199037/out --- PASS: TestMinimalIPv6NoSubnetPrefix (0.23s) === RUN TestMinimalWarmPool I0713 10:15:14.631189 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:14.631271 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:14.631298 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:14.631332 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:14.631358 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:14.631392 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:14.631444 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:14.631463 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:14.631487 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:14.631504 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:14.647179 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:14.649177 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.649232 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.721598 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:14.873103 13927 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0713 10:15:14.873935 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:14.873967 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:14.874064 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:14.875640 13927 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0713 10:15:14.882806 13927 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0713 10:15:14.882988 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.883212 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.884013 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.884811 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.884905 13927 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0713 10:15:14.885245 13927 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0713 10:15:14.885453 13927 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0713 10:15:14.890752 13927 target.go:116] Terraform output is in /tmp/test2252462236/out --- PASS: TestMinimalWarmPool (0.26s) === RUN TestMinimalEtcd I0713 10:15:14.894587 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:14.894717 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:14.894762 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:14.894822 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:14.894866 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:14.894920 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:14.895007 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:14.895040 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:14.895080 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:14.895110 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:14.923508 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:14.926796 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.926893 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:14.998190 13927 zones.go:93] ListHostedZonesPages { } W0713 10:15:15.064834 13927 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0713 10:15:15.064876 13927 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 10 W0713 10:15:15.066005 13927 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0713 10:15:15.073647 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:15.074207 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:15.074239 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:15.076575 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:15.079962 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:15.082984 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:15.083262 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.083475 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.085066 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.085613 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.085713 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:15.086180 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:15.086400 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:15.090026 13927 target.go:116] Terraform output is in /tmp/test2306109339/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0713 10:15:15.092207 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:15.092290 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:15.092332 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:15.092363 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:15.092394 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:15.092425 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:15.092478 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:15.092499 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:15.092522 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:15.092540 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:15.118322 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:15.120767 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.120837 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.191111 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:15.301699 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:15.302009 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:15.302030 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:15.302116 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:15.303968 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:15.307170 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:15.307439 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.307594 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.308917 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.309005 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.309103 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:15.309342 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:15.309470 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:15.313837 13927 target.go:116] Terraform output is in /tmp/test2008634269/out --- PASS: TestMinimalGp3 (0.22s) === RUN TestMinimalLongClusterName I0713 10:15:15.317272 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:15.317391 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:15.317431 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:15.317481 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:15.317526 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:15.317578 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:15.317664 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:15.317698 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:15.317740 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:15.317772 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:15.343736 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:15.349886 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.349985 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0713 10:15:15.422038 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:15.530295 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:15.530956 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:15.530984 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:15.531082 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:15.537431 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:15.541748 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:15.542153 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.542338 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.543507 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.543915 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.543998 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:15.544282 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:15.544410 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:15.548821 13927 target.go:116] Terraform output is in /tmp/test3533776400/out --- PASS: TestMinimalLongClusterName (0.23s) === RUN TestExistingSG I0713 10:15:15.552323 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:15.552463 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:15.552505 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:15.552554 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:15.552592 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:15.552636 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:15.552759 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:15.552791 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:15.552832 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:15.552857 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:15.595052 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:15.598139 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.598222 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.598272 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.598320 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.669821 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:15.767621 13927 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0713 10:15:15.768158 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:15.768197 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:15.768299 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:15.769928 13927 executor.go:111] Tasks: 53 done / 146 total; 22 can run I0713 10:15:15.772168 13927 executor.go:111] Tasks: 75 done / 146 total; 50 can run I0713 10:15:15.772738 13927 executor.go:111] Tasks: 125 done / 146 total; 5 can run W0713 10:15:15.772778 13927 api.go:53] PageSize not implemented W0713 10:15:15.772823 13927 api.go:53] PageSize not implemented W0713 10:15:15.772866 13927 api.go:53] PageSize not implemented W0713 10:15:15.772973 13927 api.go:53] PageSize not implemented I0713 10:15:15.780016 13927 executor.go:111] Tasks: 130 done / 146 total; 8 can run I0713 10:15:15.780209 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.780400 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.780593 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.780784 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.781807 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.782284 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.782802 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.782884 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.782968 13927 executor.go:111] Tasks: 138 done / 146 total; 4 can run I0713 10:15:15.783237 13927 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0713 10:15:15.783561 13927 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0713 10:15:15.791083 13927 target.go:116] Terraform output is in /tmp/test1809977968/out --- PASS: TestExistingSG (0.24s) === RUN TestBastionAdditionalUserData I0713 10:15:15.795197 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:15.795321 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:15.795363 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:15.795411 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:15.795454 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:15.795501 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:15.795582 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:15.795612 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:15.795663 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:15.795694 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:15.824586 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:15.827674 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.827751 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.827810 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.897842 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:15.972396 13927 executor.go:111] Tasks: 0 done / 121 total; 46 can run I0713 10:15:15.977053 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:15.977080 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:15.977145 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:15.978585 13927 executor.go:111] Tasks: 46 done / 121 total; 26 can run I0713 10:15:15.980796 13927 executor.go:111] Tasks: 72 done / 121 total; 31 can run I0713 10:15:15.981234 13927 executor.go:111] Tasks: 103 done / 121 total; 5 can run W0713 10:15:15.981296 13927 api.go:53] PageSize not implemented W0713 10:15:15.981335 13927 api.go:53] PageSize not implemented W0713 10:15:15.981376 13927 api.go:53] PageSize not implemented I0713 10:15:15.984047 13927 executor.go:111] Tasks: 108 done / 121 total; 7 can run I0713 10:15:15.984292 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.984495 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.984747 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.985549 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.985583 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.985972 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:15.986041 13927 executor.go:111] Tasks: 115 done / 121 total; 3 can run I0713 10:15:15.986264 13927 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0713 10:15:15.986410 13927 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0713 10:15:15.990079 13927 target.go:116] Terraform output is in /tmp/test1123422624/out --- PASS: TestBastionAdditionalUserData (0.20s) === RUN TestPrivateWeave I0713 10:15:15.992330 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:15.992412 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:15.992436 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:15.992465 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:15.992488 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:15.992517 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:15.992565 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:15.992583 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:15.992607 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:15.992624 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:16.016806 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:16.020266 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.020350 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.020412 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.092456 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:16.178079 13927 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0713 10:15:16.181202 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:16.181230 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:16.181311 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:16.219034 13927 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0713 10:15:16.222675 13927 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0713 10:15:16.224134 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.224296 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.226056 13927 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0713 10:15:16.226102 13927 api.go:53] PageSize not implemented W0713 10:15:16.227642 13927 api.go:53] PageSize not implemented I0713 10:15:16.233898 13927 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0713 10:15:16.234626 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.235561 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.236562 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.237014 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.237090 13927 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0713 10:15:16.237285 13927 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0713 10:15:16.237426 13927 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0713 10:15:16.242376 13927 target.go:116] Terraform output is in /tmp/test3727569750/out --- PASS: TestPrivateWeave (0.25s) === RUN TestPrivateFlannel I0713 10:15:16.246099 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:16.246258 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:16.246315 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:16.246428 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:16.246497 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:16.246580 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:16.246624 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:16.246657 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:16.246697 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:16.246727 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:16.267600 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:16.269621 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.269682 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.269719 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.339526 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:16.480147 13927 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0713 10:15:16.480779 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:16.480807 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:16.480908 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:16.486842 13927 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0713 10:15:16.488845 13927 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0713 10:15:16.489103 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.489285 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.489347 13927 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0713 10:15:16.489405 13927 api.go:53] PageSize not implemented W0713 10:15:16.489454 13927 api.go:53] PageSize not implemented I0713 10:15:16.492388 13927 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0713 10:15:16.492488 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.492625 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.493389 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.494149 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.494223 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:16.494386 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:16.494503 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:16.499917 13927 target.go:116] Terraform output is in /tmp/test985843512/out --- PASS: TestPrivateFlannel (0.26s) === RUN TestPrivateCalico I0713 10:15:16.503257 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:16.503366 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:16.503402 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:16.503476 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:16.503512 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:16.503559 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:16.503594 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:16.503622 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:16.503660 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:16.503689 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:16.521800 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:16.524836 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.524913 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.524965 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:16.593953 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:17.001881 13927 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0713 10:15:17.002524 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:17.002557 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:17.002664 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:17.007949 13927 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0713 10:15:17.010886 13927 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0713 10:15:17.011310 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.011450 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.011520 13927 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0713 10:15:17.011545 13927 api.go:53] PageSize not implemented W0713 10:15:17.011565 13927 api.go:53] PageSize not implemented I0713 10:15:17.014413 13927 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0713 10:15:17.014577 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.014743 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.016070 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.016175 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.016248 13927 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0713 10:15:17.016632 13927 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0713 10:15:17.016847 13927 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0713 10:15:17.021242 13927 target.go:116] Terraform output is in /tmp/test790175954/out --- PASS: TestPrivateCalico (0.52s) === RUN TestPrivateCilium I0713 10:15:17.024587 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:17.024697 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:17.024736 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:17.024780 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:17.024814 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:17.024856 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:17.024936 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:17.024966 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:17.024999 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:17.025022 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:17.047288 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:17.049367 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.049423 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.049457 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.122143 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:17.246859 13927 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0713 10:15:17.247400 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:17.247422 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:17.247501 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:17.249289 13927 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0713 10:15:17.251302 13927 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0713 10:15:17.251730 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.251833 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.251897 13927 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0713 10:15:17.251943 13927 api.go:53] PageSize not implemented W0713 10:15:17.252013 13927 api.go:53] PageSize not implemented I0713 10:15:17.258855 13927 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0713 10:15:17.259058 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.259255 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.260133 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.261756 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.261845 13927 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0713 10:15:17.262068 13927 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0713 10:15:17.262197 13927 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0713 10:15:17.268163 13927 target.go:116] Terraform output is in /tmp/test1226218211/out --- PASS: TestPrivateCilium (0.25s) === RUN TestPrivateCilium2 I0713 10:15:17.271596 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:17.271710 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:17.271749 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:17.271833 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:17.271874 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:17.271924 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:17.271963 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:17.271994 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:17.272034 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:17.272068 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:17.299395 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:17.302634 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.302713 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.302767 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.373297 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:17.881942 13927 executor.go:111] Tasks: 0 done / 124 total; 51 can run I0713 10:15:17.882383 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:17.882413 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:17.882502 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:17.888780 13927 executor.go:111] Tasks: 51 done / 124 total; 26 can run I0713 10:15:17.891067 13927 executor.go:111] Tasks: 77 done / 124 total; 32 can run I0713 10:15:17.891397 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.891548 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.891641 13927 executor.go:111] Tasks: 109 done / 124 total; 5 can run W0713 10:15:17.891691 13927 api.go:53] PageSize not implemented W0713 10:15:17.891769 13927 api.go:53] PageSize not implemented I0713 10:15:17.894494 13927 executor.go:111] Tasks: 114 done / 124 total; 6 can run I0713 10:15:17.894688 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.894829 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.895720 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.896219 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.896281 13927 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0713 10:15:17.896451 13927 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0713 10:15:17.896557 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:17.900528 13927 target.go:116] Terraform output is in /tmp/test2155573871/out --- PASS: TestPrivateCilium2 (0.64s) === RUN TestPrivateCiliumAdvanced I0713 10:15:17.908203 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:17.908303 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:17.908336 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:17.908369 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:17.908394 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:17.908425 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:17.908481 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:17.908498 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:17.908520 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:17.908538 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:17.928424 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:17.930523 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.930579 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:17.930614 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.000297 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:18.112103 13927 executor.go:111] Tasks: 0 done / 126 total; 53 can run I0713 10:15:18.112784 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:18.112806 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:18.112868 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:18.114509 13927 executor.go:111] Tasks: 53 done / 126 total; 26 can run I0713 10:15:18.117680 13927 executor.go:111] Tasks: 79 done / 126 total; 32 can run I0713 10:15:18.117861 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.117962 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.118177 13927 executor.go:111] Tasks: 111 done / 126 total; 5 can run W0713 10:15:18.118280 13927 api.go:53] PageSize not implemented W0713 10:15:18.118320 13927 api.go:53] PageSize not implemented I0713 10:15:18.121528 13927 executor.go:111] Tasks: 116 done / 126 total; 6 can run I0713 10:15:18.121626 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.121779 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.123953 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.126256 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.126331 13927 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0713 10:15:18.126525 13927 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0713 10:15:18.126673 13927 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0713 10:15:18.130434 13927 target.go:116] Terraform output is in /tmp/test1193662859/out --- PASS: TestPrivateCiliumAdvanced (0.22s) === RUN TestPrivateCiliumENI I0713 10:15:18.132865 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:18.132942 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:18.132967 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:18.132993 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:18.133020 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:18.133051 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:18.133101 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:18.133118 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:18.133140 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:18.133159 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:18.150096 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:18.152064 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.152118 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.152154 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.221068 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:18.322302 13927 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0713 10:15:18.322561 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:18.322585 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:18.322675 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:18.324045 13927 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0713 10:15:18.326049 13927 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0713 10:15:18.326259 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.326359 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.326542 13927 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0713 10:15:18.326595 13927 api.go:53] PageSize not implemented W0713 10:15:18.326713 13927 api.go:53] PageSize not implemented I0713 10:15:18.329305 13927 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0713 10:15:18.329405 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.329552 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.330213 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.330969 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.331035 13927 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0713 10:15:18.331206 13927 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0713 10:15:18.331306 13927 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0713 10:15:18.334825 13927 target.go:116] Terraform output is in /tmp/test14639227/out --- PASS: TestPrivateCiliumENI (0.20s) === RUN TestPrivateCanal I0713 10:15:18.337125 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:18.337201 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:18.337224 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:18.337276 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:18.337301 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:18.337333 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:18.337358 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:18.337376 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:18.337401 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:18.337420 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:18.357791 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:18.359776 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.359834 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.359871 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.429160 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:18.728519 13927 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0713 10:15:18.728995 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:18.729028 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:18.729120 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:18.730817 13927 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0713 10:15:18.733414 13927 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0713 10:15:18.733876 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.733974 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.734027 13927 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0713 10:15:18.734075 13927 api.go:53] PageSize not implemented W0713 10:15:18.734160 13927 api.go:53] PageSize not implemented I0713 10:15:18.736941 13927 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0713 10:15:18.737049 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.737175 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.738041 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.738599 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.738665 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:18.738830 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:18.738937 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:18.742642 13927 target.go:116] Terraform output is in /tmp/test4113644735/out --- PASS: TestPrivateCanal (0.41s) === RUN TestPrivateKopeio I0713 10:15:18.745482 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:18.745559 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:18.745584 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:18.745614 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:18.745639 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:18.745683 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:18.745733 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:18.745749 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:18.745771 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:18.745791 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:18.767290 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:18.769331 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.769392 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.769426 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.839324 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:18.936038 13927 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0713 10:15:18.936769 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:18.936798 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:18.936896 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:18.938520 13927 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0713 10:15:18.943930 13927 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0713 10:15:18.944465 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.944622 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.944718 13927 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0713 10:15:18.944772 13927 api.go:53] PageSize not implemented W0713 10:15:18.944924 13927 api.go:53] PageSize not implemented I0713 10:15:18.949274 13927 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0713 10:15:18.949413 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.949515 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.950332 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.950971 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.951042 13927 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0713 10:15:18.951246 13927 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0713 10:15:18.951357 13927 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0713 10:15:18.955440 13927 target.go:116] Terraform output is in /tmp/test3061446211/out --- PASS: TestPrivateKopeio (0.21s) === RUN TestUnmanaged I0713 10:15:18.957712 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:18.957793 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:18.957817 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:18.957845 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:18.957869 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:18.957900 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:18.957948 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:18.957964 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:18.957987 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:18.958004 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:18.975729 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:18.977658 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.977712 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:18.977746 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.047011 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:19.119544 13927 executor.go:111] Tasks: 0 done / 110 total; 45 can run I0713 10:15:19.123284 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:19.123326 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:19.123477 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:19.125461 13927 executor.go:111] Tasks: 45 done / 110 total; 23 can run I0713 10:15:19.127528 13927 executor.go:111] Tasks: 68 done / 110 total; 29 can run I0713 10:15:19.127744 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.127899 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.127969 13927 executor.go:111] Tasks: 97 done / 110 total; 4 can run W0713 10:15:19.128002 13927 api.go:53] PageSize not implemented W0713 10:15:19.128096 13927 api.go:53] PageSize not implemented I0713 10:15:19.130862 13927 executor.go:111] Tasks: 101 done / 110 total; 5 can run I0713 10:15:19.131027 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.131177 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.132234 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.132714 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.132786 13927 executor.go:111] Tasks: 106 done / 110 total; 2 can run I0713 10:15:19.132958 13927 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0713 10:15:19.133063 13927 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0713 10:15:19.136413 13927 target.go:116] Terraform output is in /tmp/test560535260/out --- PASS: TestUnmanaged (0.18s) === RUN TestPrivateSharedSubnet I0713 10:15:19.138530 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:19.138608 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:19.138633 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:19.138659 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:19.138690 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:19.138727 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:19.138777 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:19.138796 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:19.138818 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:19.138839 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:19.156860 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:19.160409 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.160471 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.160507 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.231627 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:19.302475 13927 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0713 10:15:19.302898 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:19.302928 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:19.303008 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:19.304998 13927 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0713 10:15:19.305309 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:19.307449 13927 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0713 10:15:19.307879 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.307992 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.308057 13927 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0713 10:15:19.308082 13927 api.go:53] PageSize not implemented W0713 10:15:19.308182 13927 api.go:53] PageSize not implemented I0713 10:15:19.311100 13927 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0713 10:15:19.311203 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.312056 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.314708 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.316109 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.316227 13927 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0713 10:15:19.316456 13927 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0713 10:15:19.316586 13927 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0713 10:15:19.321770 13927 target.go:116] Terraform output is in /tmp/test146213183/out --- PASS: TestPrivateSharedSubnet (0.19s) === RUN TestPrivateSharedIP I0713 10:15:19.325450 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:19.325563 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:19.325598 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:19.325656 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:19.325700 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:19.325748 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:19.325829 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:19.325859 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:19.325906 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:19.325934 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:19.355073 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:19.358272 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.358353 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.358414 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.427818 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:19.498668 13927 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0713 10:15:19.499124 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:19.499144 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:19.499221 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:19.505470 13927 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0713 10:15:19.505742 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:19.508425 13927 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0713 10:15:19.508809 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.508910 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.508968 13927 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0713 10:15:19.509010 13927 api.go:53] PageSize not implemented W0713 10:15:19.509099 13927 api.go:53] PageSize not implemented I0713 10:15:19.512214 13927 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0713 10:15:19.512457 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.512626 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.513904 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.514537 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.514617 13927 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0713 10:15:19.514834 13927 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0713 10:15:19.514953 13927 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0713 10:15:19.518415 13927 target.go:116] Terraform output is in /tmp/test1673662319/out --- PASS: TestPrivateSharedIP (0.20s) === RUN TestPrivateDns1 I0713 10:15:19.520555 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:19.520630 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:19.520655 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:19.520681 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:19.520704 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:19.520736 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:19.520785 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:19.520803 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:19.520828 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:19.520845 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:19.539526 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.539586 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.539621 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.675828 13927 executor.go:111] Tasks: 0 done / 120 total; 46 can run I0713 10:15:19.678166 13927 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0713 10:15:19.678456 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "internal.example.com" I0713 10:15:19.678494 13927 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0713 10:15:19.678569 13927 dnszone.go:248] Existing zone "internal.example.com." found; will configure TF to reuse I0713 10:15:19.678580 13927 dnszone.go:271] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0713 10:15:19.678618 13927 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0713 10:15:19.678862 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.678967 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.682541 13927 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0713 10:15:19.682687 13927 api.go:53] PageSize not implemented W0713 10:15:19.682775 13927 api.go:53] PageSize not implemented I0713 10:15:19.686527 13927 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0713 10:15:19.686641 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.686778 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.688039 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.689073 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.689195 13927 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0713 10:15:19.689425 13927 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0713 10:15:19.689526 13927 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0713 10:15:19.693571 13927 target.go:116] Terraform output is in /tmp/test3816835/out --- PASS: TestPrivateDns1 (0.18s) === RUN TestPrivateDns2 I0713 10:15:19.695891 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:19.695974 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:19.696000 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:19.696032 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:19.696058 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:19.696090 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:19.696139 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:19.696156 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:19.696179 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:19.696195 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:19.714990 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.715058 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.715092 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.841062 13927 executor.go:111] Tasks: 0 done / 116 total; 44 can run I0713 10:15:19.847013 13927 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0713 10:15:19.847171 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:19.847336 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "private.example.com" I0713 10:15:19.847363 13927 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0713 10:15:19.847413 13927 dnszone.go:248] Existing zone "private.example.com." found; will configure TF to reuse I0713 10:15:19.847427 13927 dnszone.go:262] VPC "vpc-12345678" already associated with zone "private.example.com." I0713 10:15:19.847463 13927 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0713 10:15:19.847652 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.847815 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.849654 13927 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0713 10:15:19.849753 13927 api.go:53] PageSize not implemented W0713 10:15:19.849790 13927 api.go:53] PageSize not implemented I0713 10:15:19.852629 13927 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0713 10:15:19.852749 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.852848 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.853745 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.854164 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.854234 13927 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0713 10:15:19.854417 13927 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0713 10:15:19.854527 13927 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0713 10:15:19.858272 13927 target.go:116] Terraform output is in /tmp/test1001853593/out --- PASS: TestPrivateDns2 (0.16s) === RUN TestDiscoveryFeatureGate I0713 10:15:19.860476 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:19.860551 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:19.860580 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:19.860635 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:19.860661 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:19.860695 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:19.860720 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:19.860737 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:19.860759 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:19.860775 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:19.876032 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:19.877911 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.877968 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:19.950308 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:20.114212 13927 executor.go:111] Tasks: 0 done / 99 total; 51 can run I0713 10:15:20.114713 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:20.114739 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:20.114831 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:20.119608 13927 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0713 10:15:20.128372 13927 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0713 10:15:20.128676 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.128953 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.129822 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.131740 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.131851 13927 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0713 10:15:20.132141 13927 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0713 10:15:20.132323 13927 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0713 10:15:20.135766 13927 target.go:116] Terraform output is in /tmp/test2373397290/out --- PASS: TestDiscoveryFeatureGate (0.28s) === RUN TestVFSServiceAccountIssuerDiscovery I0713 10:15:20.139103 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:20.139208 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:20.139239 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:20.139273 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:20.139302 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:20.139336 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:20.139393 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:20.139412 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:20.139436 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:20.139454 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:20.154970 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:20.156820 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.156878 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.227281 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:20.300218 13927 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0713 10:15:20.300890 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:20.300917 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:20.300998 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:20.302527 13927 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0713 10:15:20.306562 13927 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0713 10:15:20.306723 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.306833 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.307731 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.308180 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.308246 13927 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0713 10:15:20.308494 13927 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0713 10:15:20.308688 13927 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0713 10:15:20.311729 13927 target.go:116] Terraform output is in /tmp/test3112563662/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.17s) === RUN TestAWSLBController I0713 10:15:20.314158 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:20.314243 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:20.314271 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:20.314298 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:20.314320 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:20.314351 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:20.314403 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:20.314421 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:20.314444 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:20.314460 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:20.333810 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:20.335717 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.335822 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.407038 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:20.884460 13927 executor.go:111] Tasks: 0 done / 96 total; 49 can run I0713 10:15:20.885069 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:20.885099 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:20.885191 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:20.891360 13927 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0713 10:15:20.896487 13927 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0713 10:15:20.896926 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.897148 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.898491 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.898759 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.898839 13927 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0713 10:15:20.899082 13927 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0713 10:15:20.899212 13927 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0713 10:15:20.902744 13927 target.go:116] Terraform output is in /tmp/test1339581907/out --- PASS: TestAWSLBController (0.59s) === RUN TestManyAddons I0713 10:15:20.907576 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:20.907670 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:20.907701 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:20.907733 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:20.907761 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:20.907793 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:20.907850 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:20.907867 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:20.907890 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:20.907909 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:20.928155 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:20.930285 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:20.930347 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.000819 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:21.598533 13927 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0713 10:15:21.598890 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:21.598916 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:21.598999 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:21.600971 13927 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0713 10:15:21.604005 13927 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0713 10:15:21.604338 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.604482 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.608175 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.610089 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.610184 13927 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0713 10:15:21.610446 13927 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0713 10:15:21.610644 13927 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0713 10:15:21.614510 13927 target.go:116] Terraform output is in /tmp/test4068754962/out --- PASS: TestManyAddons (0.71s) === RUN TestManyAddonsCCMIRSA I0713 10:15:21.619259 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:21.619342 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:21.619370 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:21.619431 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:21.619457 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:21.619491 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:21.619519 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:21.619537 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:21.619563 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:21.619581 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:21.636041 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:21.638247 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.638302 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:21.707612 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:22.329085 13927 executor.go:111] Tasks: 0 done / 123 total; 61 can run I0713 10:15:22.334269 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:22.334309 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:22.334401 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:22.336086 13927 executor.go:111] Tasks: 61 done / 123 total; 30 can run I0713 10:15:22.339506 13927 executor.go:111] Tasks: 91 done / 123 total; 26 can run I0713 10:15:22.339735 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:22.339916 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:22.340671 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:22.341367 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:22.341432 13927 executor.go:111] Tasks: 117 done / 123 total; 2 can run I0713 10:15:22.341640 13927 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0713 10:15:22.341811 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:22.350981 13927 target.go:116] Terraform output is in /tmp/test2806460788/out --- PASS: TestManyAddonsCCMIRSA (0.74s) === RUN TestManyAddonsCCMIRSA23 I0713 10:15:22.360933 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:22.361055 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:22.361095 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:22.361142 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:22.361183 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:22.361230 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:22.361312 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:22.361343 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:22.361384 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:22.361412 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:22.391427 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0713 10:15:22.395113 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:22.395199 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:22.468740 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:23.235604 13927 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0713 10:15:23.236222 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:23.236250 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:23.236315 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:23.239248 13927 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0713 10:15:23.244388 13927 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0713 10:15:23.244787 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:23.245015 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:23.247088 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:23.247627 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:23.247699 13927 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0713 10:15:23.247868 13927 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0713 10:15:23.248051 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:23.252730 13927 target.go:116] Terraform output is in /tmp/test570437178/out --- PASS: TestManyAddonsCCMIRSA23 (0.90s) === RUN TestManyAddonsCCMIRSA24 I0713 10:15:23.258889 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:23.258990 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:23.259019 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:23.259050 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:23.259072 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:23.259104 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:23.259159 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:23.259176 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:23.259198 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:23.259214 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:23.284140 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:23.287336 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:23.287410 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:23.358389 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:24.138663 13927 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0713 10:15:24.139780 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:24.139803 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:24.139883 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:24.141004 13927 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0713 10:15:24.144126 13927 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0713 10:15:24.144564 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:24.144656 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:24.145766 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:24.146905 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:24.147005 13927 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0713 10:15:24.147669 13927 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0713 10:15:24.148153 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:24.158498 13927 target.go:116] Terraform output is in /tmp/test1371963349/out --- PASS: TestManyAddonsCCMIRSA24 (0.91s) === RUN TestManyAddonsCCMIRSA25 I0713 10:15:24.169697 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:24.169828 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:24.169871 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:24.169967 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:24.170011 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:24.170063 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:24.170109 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:24.170140 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:24.170180 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:24.170209 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:24.197705 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:24.200834 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:24.200912 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:24.277749 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:25.025745 13927 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0713 10:15:25.026420 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:25.026463 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:25.026590 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:25.029105 13927 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0713 10:15:25.034212 13927 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0713 10:15:25.034717 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.034950 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.036306 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.037908 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.038110 13927 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0713 10:15:25.038412 13927 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0713 10:15:25.038697 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:25.051149 13927 target.go:116] Terraform output is in /tmp/test3823185236/out --- PASS: TestManyAddonsCCMIRSA25 (0.89s) === RUN TestManyAddonsCCMIRSA26 I0713 10:15:25.061017 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:25.061238 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:25.061319 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:25.061409 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:25.061486 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:25.061574 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:25.061707 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:25.061758 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:25.061833 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:25.061886 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:25.089921 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:25.093375 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.093447 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.164151 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:25.910366 13927 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0713 10:15:25.910895 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:25.910951 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:25.911082 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:25.916852 13927 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0713 10:15:25.922578 13927 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0713 10:15:25.922895 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.924049 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.926987 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.928233 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:25.928530 13927 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0713 10:15:25.928900 13927 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0713 10:15:25.929221 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:25.937059 13927 target.go:116] Terraform output is in /tmp/test3231992027/out --- PASS: TestManyAddonsCCMIRSA26 (0.89s) === RUN TestManyAddonsGCE I0713 10:15:25.947310 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:26.698834 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:26.698896 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:26.698987 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/main" W0713 10:15:26.699008 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/events" W0713 10:15:26.699023 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com" W0713 10:15:26.699072 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:26.699997 13927 executor.go:111] Tasks: 0 done / 67 total; 44 can run I0713 10:15:26.708583 13927 executor.go:111] Tasks: 44 done / 67 total; 17 can run I0713 10:15:26.712935 13927 executor.go:111] Tasks: 61 done / 67 total; 4 can run I0713 10:15:26.715311 13927 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0713 10:15:26.715516 13927 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0713 10:15:26.718250 13927 target.go:116] Terraform output is in /tmp/test4105900209/out --- PASS: TestManyAddonsGCE (0.78s) === RUN TestCCM I0713 10:15:26.726345 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:26.726443 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:26.726473 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:26.726536 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:26.726565 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:26.726600 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:26.726628 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:26.726647 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:26.726673 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:26.726690 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:26.742769 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:26.744873 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:26.744928 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:26.815157 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:27.536697 13927 executor.go:111] Tasks: 0 done / 110 total; 53 can run I0713 10:15:27.537688 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:27.537743 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:27.537873 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:27.539439 13927 executor.go:111] Tasks: 53 done / 110 total; 24 can run I0713 10:15:27.542547 13927 executor.go:111] Tasks: 77 done / 110 total; 27 can run I0713 10:15:27.542682 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.544215 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.547407 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.549093 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.549247 13927 executor.go:111] Tasks: 104 done / 110 total; 2 can run I0713 10:15:27.549583 13927 executor.go:111] Tasks: 106 done / 110 total; 4 can run I0713 10:15:27.549932 13927 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0713 10:15:27.556606 13927 target.go:116] Terraform output is in /tmp/test1325845752/out --- PASS: TestCCM (0.84s) === RUN TestExternalDNS I0713 10:15:27.566499 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:27.566633 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:27.566677 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:27.566726 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:27.566767 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:27.566819 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:27.566911 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:27.566940 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:27.566981 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:27.567009 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:27.592685 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:27.595482 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.595563 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.665201 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:27.745720 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:27.746201 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:27.746221 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:27.746311 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:27.752579 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:27.757263 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:27.757503 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.757640 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.758892 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.760021 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.760137 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:27.760398 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:27.760554 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:27.764203 13927 target.go:116] Terraform output is in /tmp/test2782219659/out --- PASS: TestExternalDNS (0.20s) === RUN TestExternalDNSIRSA I0713 10:15:27.767755 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:27.767908 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:27.767954 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:27.768008 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:27.768052 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:27.768108 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:27.768196 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:27.768226 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:27.768265 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:27.768295 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:27.786654 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:27.788883 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.788983 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.862175 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:27.988117 13927 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0713 10:15:27.989428 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:27.989454 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:27.989546 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:27.991107 13927 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0713 10:15:27.994542 13927 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0713 10:15:27.994769 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.994923 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.995777 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.996316 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:27.996380 13927 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0713 10:15:27.996549 13927 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0713 10:15:27.996644 13927 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0713 10:15:27.999528 13927 target.go:116] Terraform output is in /tmp/test1013354844/out --- PASS: TestExternalDNSIRSA (0.23s) === RUN TestKarpenter I0713 10:15:28.001957 13927 featureflag.go:166] FeatureFlag "Karpenter"=true I0713 10:15:28.001993 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:28.002095 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:28.002131 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:28.002170 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:28.002206 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:28.002251 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:28.002335 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:28.002366 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:28.002401 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:28.002430 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:28.023675 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:28.025815 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.025865 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.025900 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.096409 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:28.244273 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.244370 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.244440 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.359512 13927 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0713 10:15:28.360295 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:28.360328 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:28.360414 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:28.361942 13927 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0713 10:15:28.365118 13927 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0713 10:15:28.365425 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.365638 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.365856 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.366067 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.368233 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.368430 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.369725 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.371925 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.372073 13927 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0713 10:15:28.372442 13927 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0713 10:15:28.372621 13927 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0713 10:15:28.379379 13927 target.go:116] Terraform output is in /tmp/test2696730923/out I0713 10:15:28.384834 13927 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.38s) === RUN TestSharedSubnet I0713 10:15:28.384948 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:28.385077 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:28.385118 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:28.385167 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:28.385207 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:28.385254 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:28.385342 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:28.385374 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:28.385413 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:28.385441 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:28.411600 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:28.414572 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.414654 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.486709 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:28.583485 13927 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0713 10:15:28.583895 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:28.583924 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:28.584006 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:28.586977 13927 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0713 10:15:28.587331 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:28.592145 13927 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0713 10:15:28.592537 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.592694 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.596598 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.597889 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.598023 13927 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0713 10:15:28.598299 13927 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0713 10:15:28.598435 13927 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0713 10:15:28.601324 13927 target.go:116] Terraform output is in /tmp/test2189194941/out --- PASS: TestSharedSubnet (0.22s) === RUN TestSharedVPC I0713 10:15:28.603498 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:28.603622 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:28.603663 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:28.603703 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:28.603749 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:28.603793 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:28.603847 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:28.603864 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:28.603887 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:28.603904 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:28.618418 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:28.620161 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.620218 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.690561 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:28.792623 13927 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0713 10:15:28.793271 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:28.793305 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:28.793372 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:28.795083 13927 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0713 10:15:28.795334 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:28.801548 13927 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0713 10:15:28.801933 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.802144 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.803494 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.804196 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.804290 13927 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0713 10:15:28.804531 13927 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0713 10:15:28.804662 13927 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0713 10:15:28.808463 13927 target.go:116] Terraform output is in /tmp/test3346675754/out --- PASS: TestSharedVPC (0.21s) === RUN TestSharedVPCIPv6 I0713 10:15:28.811737 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:28.811856 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:28.811896 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:28.812022 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:28.812077 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:28.812129 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:28.812170 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:28.812198 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:28.812237 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:28.812263 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:28.814320 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:28.814385 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:28.844491 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:28.844632 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:28.844746 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:28.848941 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.849042 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:28.932740 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:29.050949 13927 executor.go:111] Tasks: 0 done / 121 total; 45 can run I0713 10:15:29.051624 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:29.051652 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:29.051744 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:29.056468 13927 executor.go:111] Tasks: 45 done / 121 total; 25 can run I0713 10:15:29.057109 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:29.057279 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:29.059132 13927 executor.go:111] Tasks: 70 done / 121 total; 33 can run I0713 10:15:29.059572 13927 executor.go:111] Tasks: 103 done / 121 total; 6 can run I0713 10:15:29.059610 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:29.059661 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:29.059741 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:29.059784 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:29.062503 13927 executor.go:111] Tasks: 109 done / 121 total; 8 can run I0713 10:15:29.062711 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.062912 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.064286 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.066016 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.066132 13927 executor.go:111] Tasks: 117 done / 121 total; 2 can run I0713 10:15:29.066378 13927 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0713 10:15:29.066534 13927 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0713 10:15:29.070756 13927 target.go:116] Terraform output is in /tmp/test2903235237/out --- PASS: TestSharedVPCIPv6 (0.26s) === RUN TestExistingIAM I0713 10:15:29.073523 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:29.073618 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:29.073649 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:29.073681 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:29.073708 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:29.073745 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:29.073798 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:29.073816 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:29.073840 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:29.073858 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:29.093931 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:29.096023 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.096081 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.096119 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.096162 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.165796 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:29.251441 13927 executor.go:111] Tasks: 0 done / 101 total; 51 can run I0713 10:15:29.251891 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:29.251925 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:29.252019 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:29.253776 13927 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0713 10:15:29.258691 13927 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0713 10:15:29.258886 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.259100 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.259270 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.259431 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.260295 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.260659 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.261101 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.261481 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.261572 13927 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0713 10:15:29.261830 13927 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0713 10:15:29.261993 13927 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0713 10:15:29.265499 13927 target.go:116] Terraform output is in /tmp/test2581743121/out --- PASS: TestExistingIAM (0.19s) === RUN TestPhaseNetwork I0713 10:15:29.267900 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:29.267980 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:29.268011 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:29.268043 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:29.268068 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:29.268102 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:29.268128 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:29.268145 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:29.268171 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:29.268189 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:29.289059 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:29.290913 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.290969 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.291003 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.365032 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:29.435773 13927 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0713 10:15:29.436119 13927 executor.go:111] Tasks: 46 done / 119 total; 26 can run I0713 10:15:29.436430 13927 executor.go:111] Tasks: 72 done / 119 total; 32 can run I0713 10:15:29.436696 13927 executor.go:111] Tasks: 104 done / 119 total; 5 can run I0713 10:15:29.436819 13927 executor.go:111] Tasks: 109 done / 119 total; 6 can run I0713 10:15:29.436900 13927 executor.go:111] Tasks: 115 done / 119 total; 2 can run I0713 10:15:29.436928 13927 executor.go:111] Tasks: 117 done / 119 total; 2 can run I0713 10:15:29.436949 13927 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0713 10:15:29.437535 13927 target.go:116] Terraform output is in /tmp/test386069577/out --- PASS: TestPhaseNetwork (0.17s) === RUN TestExternalLoadBalancer I0713 10:15:29.437855 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:29.437936 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:29.437962 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:29.437992 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:29.438017 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:29.438047 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:29.438102 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:29.438121 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:29.438145 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:29.438162 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:29.453159 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:29.455001 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.455066 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.525390 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:29.600828 13927 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0713 10:15:29.605174 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:29.605215 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:29.605305 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:29.607395 13927 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0713 10:15:29.610783 13927 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0713 10:15:29.610997 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.611197 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.612061 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.612388 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.612450 13927 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0713 10:15:29.612612 13927 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0713 10:15:29.612727 13927 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0713 10:15:29.615576 13927 target.go:116] Terraform output is in /tmp/test1430725501/out --- PASS: TestExternalLoadBalancer (0.18s) === RUN TestPhaseIAM integration_test.go:920: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:928: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0713 10:15:29.617926 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:29.618013 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:29.618040 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:29.618070 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:29.618094 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:29.618126 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:29.618179 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:29.618196 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:29.618220 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:29.618237 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:29.638299 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:29.640360 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.640415 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.640448 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.640481 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.640511 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.640538 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.640571 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.711550 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:29.819897 13927 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0713 10:15:29.820612 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:29.820642 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:29.820727 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:29.821969 13927 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0713 10:15:29.826868 13927 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0713 10:15:29.827177 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.827284 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.827447 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.827637 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.829712 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.830264 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.830713 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.830798 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.830941 13927 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0713 10:15:29.831761 13927 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0713 10:15:29.831998 13927 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0713 10:15:29.839531 13927 target.go:116] Terraform output is in /tmp/test3006876513/out --- PASS: TestMixedInstancesASG (0.23s) === RUN TestMixedInstancesSpotASG I0713 10:15:29.844029 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:29.844158 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:29.844198 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:29.844250 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:29.844288 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:29.844333 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:29.844412 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:29.844440 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:29.844475 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:29.844500 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:29.877610 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:29.879682 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.879749 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.879784 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.879816 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.879847 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.879874 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.879899 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:29.950707 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:30.027445 13927 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0713 10:15:30.030789 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:30.030845 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:30.030979 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:30.036391 13927 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0713 10:15:30.039841 13927 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0713 10:15:30.040011 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.040305 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.040473 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.040691 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.040905 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.041710 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.042232 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.043030 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.043135 13927 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0713 10:15:30.043388 13927 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0713 10:15:30.043552 13927 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0713 10:15:30.047724 13927 target.go:116] Terraform output is in /tmp/test3233895913/out --- PASS: TestMixedInstancesSpotASG (0.21s) === RUN TestAdditionalObjects I0713 10:15:30.050576 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:30.050657 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:30.050684 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:30.050714 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:30.050739 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:30.050771 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:30.050822 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:30.050839 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:30.050868 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:30.050885 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:30.072897 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0713 10:15:30.076737 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.076833 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.148294 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:30.293310 13927 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0713 10:15:30.293879 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:30.293908 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:30.293972 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:30.295379 13927 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0713 10:15:30.298478 13927 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0713 10:15:30.298734 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.298901 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.300106 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.300555 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.300614 13927 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0713 10:15:30.300821 13927 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0713 10:15:30.300928 13927 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0713 10:15:30.305103 13927 target.go:116] Terraform output is in /tmp/test462647277/out --- PASS: TestAdditionalObjects (0.26s) === RUN TestContainerd I0713 10:15:30.308275 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:30.308381 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:30.308416 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:30.308457 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:30.308493 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:30.308538 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:30.308612 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:30.308641 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:30.308678 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:30.308704 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:30.331692 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:30.334344 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.334423 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.403822 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:30.478127 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:30.478528 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:30.478573 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:30.478670 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:30.480397 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:30.484074 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:30.484275 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.484464 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.485345 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.485809 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.485888 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:30.486070 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:30.486198 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:30.488849 13927 target.go:116] Terraform output is in /tmp/test1971591198/out --- PASS: TestContainerd (0.18s) === RUN TestContainerdCustom I0713 10:15:30.490845 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:30.490929 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:30.490957 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:30.490998 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:30.491023 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:30.491060 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:30.491113 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:30.491141 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:30.491174 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:30.491192 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:30.508692 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:30.510621 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.510696 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.581746 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:30.673618 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:30.675634 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:30.675672 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:30.675744 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:30.680353 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:30.683556 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:30.683983 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.684078 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.685492 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.686058 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.686174 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:30.686463 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:30.686594 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:30.691503 13927 target.go:116] Terraform output is in /tmp/test2573060763/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I0713 10:15:30.695457 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:30.695585 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:30.695627 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:30.695692 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:30.695736 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:30.695787 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:30.695868 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:30.695899 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:30.695938 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:30.695968 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:30.721395 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:30.724127 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.724202 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.795960 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:30.871705 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:30.872408 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:30.872430 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:30.872514 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:30.874907 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:30.878604 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:30.878909 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.879063 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.880687 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.880751 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.880832 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:30.881073 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:30.881210 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:30.884242 13927 target.go:116] Terraform output is in /tmp/test2649951735/out --- PASS: TestDockerCustom (0.19s) === RUN TestAPIServerNodes I0713 10:15:30.886494 13927 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0713 10:15:30.886512 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:30.886591 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:30.886619 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:30.886652 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:30.886680 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:30.886715 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:30.886770 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:30.886792 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:30.886818 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:30.886837 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:30.907062 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.17 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.17 ********************************************************************************* I0713 10:15:30.909086 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.909144 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:30.909181 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.517453 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:31.627276 13927 executor.go:111] Tasks: 0 done / 100 total; 47 can run I0713 10:15:31.627975 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:31.627998 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:31.628056 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:31.629424 13927 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0713 10:15:31.633478 13927 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0713 10:15:31.633752 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.633904 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.634068 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.634655 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.635238 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.635473 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.635565 13927 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0713 10:15:31.635805 13927 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0713 10:15:31.635986 13927 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0713 10:15:31.640511 13927 target.go:116] Terraform output is in /tmp/test1718682193/out I0713 10:15:31.643828 13927 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.76s) === RUN TestNTHIMDSProcessor I0713 10:15:31.643923 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:31.644026 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:31.644059 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:31.644097 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:31.644132 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:31.644175 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:31.644250 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:31.644278 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:31.644316 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:31.644341 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:31.667737 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:31.670552 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.670630 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.743018 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:31.852607 13927 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0713 10:15:31.853138 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:31.853169 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:31.853232 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:31.854877 13927 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0713 10:15:31.857675 13927 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0713 10:15:31.857800 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.858075 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.858679 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.859525 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.859596 13927 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0713 10:15:31.859782 13927 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0713 10:15:31.859942 13927 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0713 10:15:31.862850 13927 target.go:116] Terraform output is in /tmp/test3320443841/out --- PASS: TestNTHIMDSProcessor (0.22s) === RUN TestNTHIMDSProcessorIRSA I0713 10:15:31.865313 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:31.865390 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:31.865415 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:31.865443 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:31.865467 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:31.865498 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:31.865547 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:31.865564 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:31.865593 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:31.865610 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:31.884957 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:31.886868 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.886929 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:31.960681 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:32.054211 13927 executor.go:111] Tasks: 0 done / 95 total; 48 can run I0713 10:15:32.057139 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:32.057179 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:32.057311 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:32.060335 13927 executor.go:111] Tasks: 48 done / 95 total; 22 can run I0713 10:15:32.063635 13927 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0713 10:15:32.063964 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.064165 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.065027 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.065345 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.065412 13927 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0713 10:15:32.065574 13927 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0713 10:15:32.065724 13927 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0713 10:15:32.068746 13927 target.go:116] Terraform output is in /tmp/test41824582/out --- PASS: TestNTHIMDSProcessorIRSA (0.21s) === RUN TestCustomIRSA I0713 10:15:32.071234 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:32.071312 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:32.071338 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:32.071365 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:32.071389 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:32.071421 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:32.071471 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:32.071488 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:32.071509 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:32.071526 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:32.088907 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:32.090922 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.090978 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.159928 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:32.575120 13927 executor.go:111] Tasks: 0 done / 100 total; 50 can run I0713 10:15:32.575836 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:32.575862 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:32.575968 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:32.576990 13927 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0713 10:15:32.580294 13927 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0713 10:15:32.580475 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.580640 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.581492 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.581884 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.581947 13927 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0713 10:15:32.582122 13927 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0713 10:15:32.582230 13927 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0713 10:15:32.589742 13927 target.go:116] Terraform output is in /tmp/test2497155388/out --- PASS: TestCustomIRSA (0.52s) === RUN TestClusterNameDigit I0713 10:15:32.594740 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:32.594829 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:32.594859 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:32.594891 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:32.594915 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:32.594946 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:32.594999 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:32.595016 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:32.595038 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:32.595057 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:32.610869 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:32.612751 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.612805 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.681916 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:32.752908 13927 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0713 10:15:32.753695 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:32.753721 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:32.753805 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:32.754750 13927 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0713 10:15:32.761135 13927 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0713 10:15:32.761422 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.761561 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.762528 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.762871 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.762945 13927 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0713 10:15:32.763118 13927 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0713 10:15:32.763225 13927 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0713 10:15:32.766418 13927 target.go:116] Terraform output is in /tmp/test1200901803/out --- PASS: TestClusterNameDigit (0.17s) === RUN TestCASPriorityExpander I0713 10:15:32.769084 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:32.769176 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:32.769207 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:32.769252 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:32.769287 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:32.769333 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:32.769395 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:32.769423 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:32.769463 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:32.769486 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:32.787080 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:32.789279 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.789347 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.789395 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:32.789449 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0713 10:15:33.418067 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:33.548893 13927 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0713 10:15:33.551888 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:33.551927 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:33.552055 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:33.555414 13927 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0713 10:15:33.560183 13927 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0713 10:15:33.560506 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.560617 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.560785 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.560984 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.561463 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.561909 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.561976 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.562865 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.562938 13927 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0713 10:15:33.563191 13927 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0713 10:15:33.563372 13927 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0713 10:15:33.566890 13927 target.go:116] Terraform output is in /tmp/test4253444485/out --- PASS: TestCASPriorityExpander (0.80s) === RUN TestCASPriorityExpanderCustom I0713 10:15:33.569490 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:33.569575 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:33.569603 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:33.569635 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:33.569658 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:33.569689 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:33.569742 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:33.569760 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:33.569783 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:33.569802 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:33.595243 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0713 10:15:33.598223 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.598292 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.598325 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.598357 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0713 10:15:33.670933 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:33.810522 13927 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0713 10:15:33.811238 13927 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0713 10:15:33.811301 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:33.811460 13927 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0713 10:15:33.814407 13927 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0713 10:15:33.822735 13927 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0713 10:15:33.822905 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.823067 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.823257 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.823454 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.823882 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.824366 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.825479 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.826007 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.826100 13927 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0713 10:15:33.826426 13927 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0713 10:15:33.826634 13927 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0713 10:15:33.832844 13927 target.go:116] Terraform output is in /tmp/test989537061/out --- PASS: TestCASPriorityExpanderCustom (0.27s) === RUN TestLifecycleMinimalAWS I0713 10:15:33.837485 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:33.837628 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:33.837674 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:33.838076 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:33.838110 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:33.838148 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:33.838182 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:33.838203 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:33.838233 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:33.838255 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:33.838296 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0713 10:15:33.856683 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:33.858634 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:33.858698 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0713 10:15:33.928819 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:34.024421 13927 iamrole.go:89] ListRoles: { } I0713 10:15:34.025332 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:34.025463 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:34.026408 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.027326 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.027543 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.028256 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.028367 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:34.028415 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.028515 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:34.028560 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:34.028640 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0713 10:15:34.028898 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0713 10:15:34.028889 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:34.029075 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:34.029663 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0713 10:15:34.029149 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:34.029777 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0713 10:15:34.031011 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:34.031059 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:34.031093 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:34.031115 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:34.031191 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:34.031218 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0713 10:15:34.031869 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:34.032072 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:34.032001 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0713 10:15:34.032163 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:34.032208 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0713 10:15:34.032229 13927 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0713 10:15:34.032152 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:34.032337 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:34.032332 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:34.032424 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:34.032473 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:34.032505 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.032594 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:34.032635 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:34.032700 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:34.032770 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:34.032807 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.032964 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.033023 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:34.033055 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.033184 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0713 10:15:34.033176 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:34.033275 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:34.033303 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:34.033333 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:34.033385 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:34.033414 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0713 10:15:34.033469 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:34.033523 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:34.033552 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:34.033483 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:34.034773 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:34.034901 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.034965 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.035070 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:34.035164 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.035276 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.035357 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.035446 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:34.035583 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.035745 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.035813 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.035882 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:34.036081 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.036172 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.036221 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.036319 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:34.036351 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.036446 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:34.036544 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I0713 10:15:34.036633 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:34.036658 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0713 10:15:34.036683 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:34.036761 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.036819 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.036890 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:34.037010 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:34.037928 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:34.038105 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:34.038198 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:34.038226 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:34.038254 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:34.038293 13927 dns.go:232] Pre-creating DNS records I0713 10:15:34.038311 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:34.041502 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:34.043187 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.043235 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0713 10:15:34.113104 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:34.217542 13927 iamrole.go:89] ListRoles: { } I0713 10:15:34.218540 13927 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0713 10:15:34.218862 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.219019 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.219220 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:34.219266 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.219449 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:34.219501 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:34.219560 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.219757 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:34.219901 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:34.220864 13927 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0713 10:15:34.220967 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.221070 13927 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0713 10:15:34.221090 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:34.221170 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.221292 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:34.221230 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0713 10:15:34.221331 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.221464 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:34.221551 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0713 10:15:34.221711 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:34.221761 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0713 10:15:34.221880 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:34.223987 13927 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0713 10:15:34.224085 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.224221 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.224333 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.224549 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.224658 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.224753 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.224842 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:34.226234 13927 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0713 10:15:34.226395 13927 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0713 10:15:34.226488 13927 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0713 10:15:34.228026 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:34.228070 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:34.228094 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:34.228114 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:34.228136 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:34.228156 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:34.228176 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:34.228197 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:34.228218 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:34.228242 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:34.228265 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:34.228282 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:34.228299 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:34.228319 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:34.228337 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:34.228357 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:34.228377 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:34.228401 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:34.228420 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:34.228438 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:34.228456 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:34.228480 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:34.228502 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:34.228522 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:34.229257 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:34.229314 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:34.229327 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:34.229353 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229389 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0713 10:15:34.229422 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229458 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229486 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229518 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229544 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229568 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0713 10:15:34.229595 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229626 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:34.229650 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0713 10:15:34.229675 13927 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0713 10:15:34.229697 13927 api.go:53] PageSize not implemented I0713 10:15:34.229710 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:34.229726 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:34.229738 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:34.229755 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:34.229768 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0713 10:15:34.229802 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:34.229831 13927 iamrole.go:89] ListRoles: { } I0713 10:15:34.229875 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:34.229894 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:34.229920 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:34.229947 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:34.229966 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:34.230256 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0713 10:15:34.230306 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0713 10:15:34.230343 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.minimal.example.com ok I0713 10:15:34.230356 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:34.230292 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0713 10:15:34.230419 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0713 10:15:34.230380 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0713 10:15:34.230465 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:34.230462 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:34.230495 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0713 10:15:34.230477 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:34.230530 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:34.230567 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:34.230599 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0713 10:15:34.230656 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:34.230688 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0713 10:15:34.230804 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:34.230813 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:34.230859 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0713 10:15:34.230891 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0713 10:15:34.230900 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:34.230921 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0713 10:15:34.230888 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:34.230940 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0713 10:15:34.230962 13927 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0713 10:15:34.230969 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:34.230983 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0713 10:15:34.231013 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:34.231056 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:34.231096 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:34.231281 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.39s) === RUN TestLifecycleMinimalOpenstack I0713 10:15:34.231420 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0713 10:15:34.236630 13927 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0713 10:15:34.427434 13927 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0713 10:15:34.427471 13927 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0713 10:15:34.436722 13927 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0713 10:15:34.445010 13927 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0713 10:15:34.478672 13927 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0713 10:15:34.482334 13927 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/07/13 10:15:34 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:34 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:34 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0713 10:15:34.485052 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:34.485450 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:34.485805 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0713 10:15:35.491735 13927 executor.go:111] Tasks: 90 done / 90 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0713 10:15:35.671786 13927 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0713 10:15:35.671814 13927 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0713 10:15:35.687612 13927 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0713 10:15:35.691043 13927 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0713 10:15:35.698078 13927 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0713 10:15:35.699666 13927 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/07/13 10:15:35 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:35 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:35 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0713 10:15:35.703002 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:35.703323 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:35.703580 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0713 10:15:35.705139 13927 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0713 10:15:35.706663 13927 delete_cluster.go:128] Looking for cloud resources to delete 2023/07/13 10:15:35 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) Port:cc8cda2e-1711-43ff-86c1-36a8691b4fd3 ok Port:2e81d01d-5bbc-4b80-92f3-254a257e539a ok ServerGroup:d197f322-c74f-4f9e-b3e8-33c89fdbb402 ok SecurityGroup:1ec29a0c-3fed-4270-92ef-10fbe64ec17a ok Instance:88a41a88-ff7a-4493-908b-55c5e716be3c ok Port:16648008-dae7-49ca-865b-7eba2c79e731 ok Network:c6092c20-eee9-4011-b67a-e07a80767911 ok SecurityGroup:85ec3ffe-4cc0-4a8e-89fa-9da2fe0aaab5 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Subnet:f925b484-29ac-42e6-9055-beac4ef57293 ok ServerGroup:aeecb358-201c-4269-92e6-04b56fc7198b ok SecurityGroup:87ae5c70-967f-4ff1-90d1-b199031ca5d3 ok Instance:ae567cb3-f8c2-452a-bdb1-4b0a560de8f8 ok Instance:df281351-b40e-460b-b50f-3d0062bb1161 ok --- PASS: TestLifecycleMinimalOpenstack (2.59s) === RUN TestLifecycleMinimalGCE I0713 10:15:36.821753 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0713 10:15:36.823559 13927 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:37.078750 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:37.078853 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:37.078986 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0713 10:15:37.079019 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0713 10:15:37.079044 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0713 10:15:37.079134 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:37.080655 13927 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0713 10:15:37.083041 13927 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0713 10:15:37.083108 13927 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0713 10:15:37.087616 13927 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0713 10:15:37.089820 13927 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0713 10:15:37.090003 13927 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0713 10:15:37.090085 13927 dns.go:232] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0713 10:15:37.347342 13927 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0713 10:15:37.347441 13927 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0713 10:15:37.347540 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0713 10:15:37.347567 13927 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0713 10:15:37.347587 13927 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0713 10:15:37.347667 13927 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0713 10:15:37.349131 13927 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0713 10:15:37.353086 13927 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0713 10:15:37.357497 13927 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0713 10:15:37.361930 13927 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0713 10:15:37.362364 13927 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0713 10:15:37.363945 13927 delete_cluster.go:128] Looking for cloud resources to delete I0713 10:15:37.363985 13927 gce.go:99] Scanning zones: [us-test1-a] I0713 10:15:37.364225 13927 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0713 10:15:37.364240 13927 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok Disk:a-etcd-events-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok Disk:a-etcd-main-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1689243337 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1689243337 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.54s) === RUN TestLifecycleFloatingIPOpenstack I0713 10:15:37.365409 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0713 10:15:37.372047 13927 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0713 10:15:37.548164 13927 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0713 10:15:37.548207 13927 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0713 10:15:37.557265 13927 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0713 10:15:37.560543 13927 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0713 10:15:37.567645 13927 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0713 10:15:37.571361 13927 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/07/13 10:15:37 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:37 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:37 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0713 10:15:37.573093 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:37.573205 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:37.573285 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0713 10:15:38.578082 13927 executor.go:111] Tasks: 92 done / 92 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0713 10:15:38.772292 13927 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0713 10:15:38.772455 13927 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0713 10:15:38.790251 13927 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0713 10:15:38.797243 13927 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0713 10:15:38.801628 13927 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0713 10:15:38.802999 13927 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/07/13 10:15:38 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:38 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/07/13 10:15:38 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0713 10:15:38.805000 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:38.805583 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0713 10:15:38.805761 13927 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0713 10:15:38.808387 13927 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0713 10:15:38.811104 13927 delete_cluster.go:128] Looking for cloud resources to delete 2023/07/13 10:15:38 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) SecurityGroup:2f209cf9-faf6-4e96-bcc3-feafe424e276 ok Subnet:c2e8e304-f224-4e1f-ba31-c8dde466d115 ok Router:6b076529-29e3-4727-98dc-e5e5ddf61f8e ok Instance:71ea0d5d-bfb7-4350-89cb-cacd3cfd4663 ok Port:c706ae55-71b1-4b55-b470-7c0e828d2a75 ok Router-IF:c2e8e304-f224-4e1f-ba31-c8dde466d115 ok SecurityGroup:b643a196-b76b-43ca-bcb0-bbff17ca6a7d ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:9dd79314-8cfc-426f-a795-93c0bb68544e ok ServerGroup:32fcdca5-7245-4aba-8c3e-30260e94f04d ok Port:655431e0-53fa-4477-bbf6-91026631e18b ok ServerGroup:fef306b7-9e67-4464-88ec-675d0410022e ok Instance:2bdb7fc4-5c32-4a4d-be26-2c0aba28d4db ok Port:b26ea513-08e4-4ae4-a721-dbd25f7b7dbd ok Instance:e1ffb09b-d0d5-498d-a29e-04eeaf9d0311 ok Network:465bdeda-0952-4b65-bce9-26e4088d3e13 ok Port:aeebbe9b-9ff3-4622-82ef-a226b2e0e9e8 ok --- PASS: TestLifecycleFloatingIPOpenstack (2.56s) === RUN TestLifecyclePrivateCalico I0713 10:15:39.921692 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:39.921802 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:39.921832 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:39.921871 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:39.921898 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:39.921929 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:39.921956 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:39.921972 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:39.921995 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:39.922012 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:39.922048 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0713 10:15:39.940037 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:39.943015 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:39.943090 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:39.943139 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.014049 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:40.353166 13927 iamrole.go:89] ListRoles: { } I0713 10:15:40.354593 13927 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0713 10:15:40.355882 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.356180 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.356321 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:40.356367 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:40.356419 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:40.356470 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:40.356528 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:40.356678 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.356841 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.356945 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:40.356992 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.357131 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0713 10:15:40.357420 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.357474 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0713 10:15:40.357636 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0713 10:15:40.357730 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0713 10:15:40.357778 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:40.357867 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:40.357910 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0713 10:15:40.357946 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:40.357973 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0713 10:15:40.358001 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.358275 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:40.358464 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:40.362955 13927 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0713 10:15:40.363184 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0713 10:15:40.363326 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0713 10:15:40.363328 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0713 10:15:40.363197 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:40.363400 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0713 10:15:40.363462 13927 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0713 10:15:40.363432 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] }] } I0713 10:15:40.363477 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatecalico-exa-hocohm", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatecalico-exa-hocohm" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0713 10:15:40.363487 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0713 10:15:40.363559 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0713 10:15:40.363543 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0713 10:15:40.363674 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0713 10:15:40.363630 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0713 10:15:40.363857 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:40.363946 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:40.364052 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0713 10:15:40.364117 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:40.364201 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:40.364275 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.364355 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0713 10:15:40.364510 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:40.364572 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.364586 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0713 10:15:40.364761 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.364862 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:40.364923 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:40.364970 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:40.365007 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0713 10:15:40.365214 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:40.365445 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0713 10:15:40.365759 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0713 10:15:40.366069 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:40.366162 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.366326 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:40.366420 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.366499 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0713 10:15:40.366579 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0713 10:15:40.366663 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:40.366878 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:40.367016 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:40.366785 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0713 10:15:40.367131 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:40.367242 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0713 10:15:40.366683 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0713 10:15:40.367439 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:40.367538 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:40.367650 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:40.367795 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:40.367864 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:40.367933 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0713 10:15:40.368034 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:40.368103 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:40.368150 13927 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0713 10:15:40.368285 13927 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0713 10:15:40.368493 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:40.368648 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:40.368951 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:40.369032 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:40.368830 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0713 10:15:40.369036 13927 api.go:53] PageSize not implemented I0713 10:15:40.369123 13927 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatecalico-exa-hocohm", SubnetMappings: [{ SubnetId: "subnet-2" }], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], Type: "network" } I0713 10:15:40.369075 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.369226 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.369342 13927 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0713 10:15:40.369394 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.369415 13927 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0713 10:15:40.369461 13927 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0713 10:15:40.369436 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.369680 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.370117 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0713 10:15:40.370279 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:40.370437 13927 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.370486 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.370672 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.370884 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:40.370931 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0713 10:15:40.371038 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.371105 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0713 10:15:40.371173 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:40.371238 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:40.371274 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.371402 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:40.371426 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" }] }] } I0713 10:15:40.371508 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0713 10:15:40.371512 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1", Port: 22, Protocol: "TCP" } W0713 10:15:40.371707 13927 api.go:53] PageSize not implemented I0713 10:15:40.371583 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:40.371732 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.371746 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.371851 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.371901 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0713 10:15:40.371953 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0713 10:15:40.372067 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:40.372124 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.372348 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.372529 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" }] }] } I0713 10:15:40.372686 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.372856 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.373048 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.373190 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0713 10:15:40.373288 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0713 10:15:40.373416 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" }] }] } I0713 10:15:40.373587 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.373750 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.373882 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:40.373988 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:40.374105 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.374246 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.374391 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.374547 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0713 10:15:40.374711 13927 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0713 10:15:40.374776 13927 api.go:53] PageSize not implemented I0713 10:15:40.374809 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:40.374855 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.374885 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:40.375059 13927 api.go:53] PageSize not implemented I0713 10:15:40.375039 13927 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:40.375029 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:40.375127 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.375254 13927 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0713 10:15:40.375435 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0713 10:15:40.375518 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:40.375592 13927 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0713 10:15:40.380309 13927 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0713 10:15:40.380417 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:40.380620 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0713 10:15:40.380790 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.380958 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.381164 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.381281 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.383835 13927 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0713 10:15:40.384135 13927 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0713 10:15:40.384292 13927 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0713 10:15:40.384351 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:40.384401 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:40.384474 13927 dns.go:232] Pre-creating DNS records I0713 10:15:40.384513 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:40.391297 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:40.395024 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.395114 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.395203 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.466399 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:40.821039 13927 iamrole.go:89] ListRoles: { } I0713 10:15:40.822398 13927 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0713 10:15:40.822702 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:40.822741 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.822881 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:40.822921 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:40.822978 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:40.823122 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.823265 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.823395 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.823515 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:40.824424 13927 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0713 10:15:40.824514 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0713 10:15:40.824614 13927 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0713 10:15:40.824637 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.824702 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0713 10:15:40.824767 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0713 10:15:40.824737 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0713 10:15:40.824808 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0713 10:15:40.824860 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0713 10:15:40.824886 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0713 10:15:40.824847 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0713 10:15:40.824970 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:40.825012 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:40.825041 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] }] } I0713 10:15:40.825130 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:40.825155 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:40.825211 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.825294 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.825371 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:40.825405 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0713 10:15:40.825503 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:40.825550 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.825671 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:40.825737 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:40.826015 13927 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0713 10:15:40.826122 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:40.826224 13927 api.go:53] PageSize not implemented I0713 10:15:40.826248 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.826255 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:40.826295 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0713 10:15:40.826334 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.826373 13927 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } W0713 10:15:40.826348 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:40.826462 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0713 10:15:40.826496 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0713 10:15:40.826468 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:40.826517 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0713 10:15:40.826559 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0713 10:15:40.826588 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0713 10:15:40.826589 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:40.826653 13927 api.go:53] PageSize not implemented I0713 10:15:40.826664 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:40.826706 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0713 10:15:40.826672 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.826750 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0713 10:15:40.826985 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.827065 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.827274 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:40.827377 13927 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0713 10:15:40.827402 13927 api.go:53] PageSize not implemented I0713 10:15:40.827442 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.827446 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:40.827523 13927 api.go:53] PageSize not implemented I0713 10:15:40.827522 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:40.827521 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:40.827558 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.827593 13927 dnsname.go:118] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0713 10:15:40.827617 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.830456 13927 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0713 10:15:40.830523 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.830606 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:40.830706 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.830902 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.831066 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.833329 13927 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0713 10:15:40.833439 13927 api.go:53] PageSize not implemented I0713 10:15:40.833467 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.833647 13927 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0713 10:15:40.833830 13927 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0713 10:15:40.837323 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:40.837417 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:40.837549 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:40.837631 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0713 10:15:40.837705 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:40.837773 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:40.838176 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:40.838270 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:40.838362 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:40.838453 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:40.838557 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:40.838659 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0713 10:15:40.838759 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0713 10:15:40.838851 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:40.838948 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:40.839071 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:40.839159 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:40.839251 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:40.839328 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:40.839410 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:40.839487 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:40.839566 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:40.839645 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:40.839726 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:40.839799 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:40.839884 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:40.839954 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:40.840033 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:40.840129 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:40.840217 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:40.840300 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:40.842045 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:40.842131 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:40.842151 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:40.842211 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842278 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0713 10:15:40.842381 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842471 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842523 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842583 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842642 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842693 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0713 10:15:40.842761 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842847 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:40.842873 13927 natgateway.go:117] DescribeNatGateways: { } I0713 10:15:40.842924 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0713 10:15:40.842971 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0713 10:15:40.843025 13927 api.go:53] PageSize not implemented I0713 10:15:40.843043 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0713 10:15:40.843095 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:40.843137 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:40.843153 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0713 10:15:40.843192 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:40.843232 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:40.843249 13927 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0713 10:15:40.843312 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:40.843373 13927 iamrole.go:89] ListRoles: { } I0713 10:15:40.843448 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:40.843500 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:40.843556 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:40.843618 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:40.843655 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0713 10:15:40.843711 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:40.843759 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0713 10:15:40.844436 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4 ok I0713 10:15:40.844475 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:bastion.privatecalico.example.com ok volume:vol-1 ok autoscaling-group:nodes.privatecalico.example.com ok I0713 10:15:40.844573 13927 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0713 10:15:40.844597 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0713 10:15:40.844634 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:40.844692 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0713 10:15:40.844678 13927 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0713 10:15:40.844701 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } load-balancer:api-privatecalico-example-0uch4k ok I0713 10:15:40.844742 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0713 10:15:40.844752 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:40.844798 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0713 10:15:40.844839 13927 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1 ok nat-gateway:nat-1 ok volume:vol-2 ok I0713 10:15:40.844858 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0713 10:15:40.844981 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0713 10:15:40.844818 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0713 10:15:40.845057 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:40.845094 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0713 10:15:40.844788 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatecalico.example.com. ok I0713 10:15:40.845135 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0713 10:15:40.845207 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0713 10:15:40.845235 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:40.845283 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0713 10:15:40.845426 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:40.845444 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok I0713 10:15:40.845494 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:40.845523 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0713 10:15:40.845612 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:40.845620 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:40.845706 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0713 10:15:40.845750 13927 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0713 10:15:40.845759 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0713 10:15:40.845779 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0713 10:15:40.845756 13927 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0713 10:15:40.845819 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0713 10:15:40.845891 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0713 10:15:40.845866 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:40.845906 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0713 10:15:40.845937 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0713 10:15:40.845956 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:40.846023 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0713 10:15:40.846104 13927 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0713 10:15:40.846124 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0713 10:15:40.846180 13927 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0713 10:15:40.846060 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 } ] } W0713 10:15:40.846239 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:40.846260 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:40.846294 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0713 10:15:40.846415 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:40.846467 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0713 10:15:40.846530 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:40.846603 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:40.846884 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.93s) === RUN TestLifecyclePrivateKopeio I0713 10:15:40.847122 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:40.847275 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:40.847311 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:40.847358 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:40.847402 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:40.847451 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:40.847489 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:40.847528 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:40.847590 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:40.847618 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:40.847669 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0713 10:15:40.870622 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:40.872863 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.872932 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.872972 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:40.943092 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.025317 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.026736 13927 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0713 10:15:41.027045 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.027672 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:41.027805 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:41.028090 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.028291 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.028810 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:41.029264 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:41.029336 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.029490 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:41.029642 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:41.029929 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:41.030098 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.030336 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.030452 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:41.030492 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:41.030829 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0713 10:15:41.030925 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0713 10:15:41.031285 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0713 10:15:41.031358 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0713 10:15:41.030664 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0713 10:15:41.031449 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0713 10:15:41.031540 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0713 10:15:41.031595 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:41.031647 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.032824 13927 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0713 10:15:41.032918 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.033037 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0713 10:15:41.033065 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0713 10:15:41.032995 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.033132 13927 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0713 10:15:41.033112 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0713 10:15:41.033205 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.033293 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:41.033274 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0713 10:15:41.033237 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:41.033267 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatekopeio-exa-d8ef8e", Port: 22, Protocol: "TCP", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-privatekopeio-exa-d8ef8e" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0713 10:15:41.033342 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0713 10:15:41.033375 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.timeout_seconds", Value: "30" },{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0713 10:15:41.033382 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:41.033389 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0713 10:15:41.033362 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0713 10:15:41.033453 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0713 10:15:41.033501 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0713 10:15:41.033516 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.033586 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:41.033619 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.033644 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.033698 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:41.033726 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.033780 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0713 10:15:41.033848 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0713 10:15:41.033856 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.034017 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:41.034072 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.034212 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I0713 10:15:41.034257 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0713 10:15:41.034404 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.034457 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.034516 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0713 10:15:41.034577 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.034578 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0713 10:15:41.034623 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0713 10:15:41.034682 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.034723 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.034550 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0713 10:15:41.034779 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:41.034821 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:41.034860 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0713 10:15:41.034987 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0713 10:15:41.034964 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0713 10:15:41.035030 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:41.035058 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0713 10:15:41.035085 13927 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] } I0713 10:15:41.035158 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0713 10:15:41.035188 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0713 10:15:41.035208 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.035257 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:41.035282 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:41.035304 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:41.035332 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.035380 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:41.035408 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:41.035430 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:41.035460 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:41.035481 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:41.035507 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0713 10:15:41.035556 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.035584 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:41.035603 13927 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0713 10:15:41.035666 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:41.035694 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.035742 13927 executor.go:111] Tasks: 76 done / 128 total; 37 can run W0713 10:15:41.035819 13927 api.go:53] PageSize not implemented I0713 10:15:41.035816 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.035906 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0713 10:15:41.035899 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.035950 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.035938 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.036020 13927 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0713 10:15:41.035931 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0713 10:15:41.036095 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.036073 13927 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatekopeio-exa-d8ef8e", SubnetMappings: [{ SubnetId: "subnet-4" },{ SubnetId: "subnet-2" }], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }], Type: "network" } I0713 10:15:41.036192 13927 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0713 10:15:41.036155 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0713 10:15:41.036245 13927 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0713 10:15:41.036349 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.036484 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" }] }] } I0713 10:15:41.036678 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0713 10:15:41.036888 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0713 10:15:41.037000 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.037102 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.037155 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0713 10:15:41.037200 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.037280 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:41.037390 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.8.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0713 10:15:41.037579 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.037702 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0713 10:15:41.037774 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.037833 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.037903 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.037966 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0713 10:15:41.037998 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.038129 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.038259 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.038379 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:41.038500 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }] } I0713 10:15:41.038589 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.038710 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:41.038795 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:41.038848 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.038985 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:41.039032 13927 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0713 10:15:41.039072 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.039134 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.039182 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0713 10:15:41.039271 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:41.039388 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.039505 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0713 10:15:41.039621 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0713 10:15:41.039745 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.039868 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:41.039902 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:41.039942 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0713 10:15:41.039977 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0713 10:15:41.040107 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0713 10:15:41.040234 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0713 10:15:41.040272 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:41.040303 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.040341 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1", Port: 22, Protocol: "TCP" } I0713 10:15:41.040371 13927 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } W0713 10:15:41.040423 13927 api.go:53] PageSize not implemented I0713 10:15:41.040436 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.040436 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:41.040485 13927 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0713 10:15:41.040528 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" }] }] } I0713 10:15:41.040694 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0713 10:15:41.040745 13927 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0713 10:15:41.040795 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0713 10:15:41.040840 13927 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0713 10:15:41.040880 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0713 10:15:41.040922 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-4" } I0713 10:15:41.040958 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:41.040999 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0713 10:15:41.041041 13927 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0713 10:15:41.041090 13927 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0713 10:15:41.041131 13927 api.go:53] PageSize not implemented I0713 10:15:41.041178 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.041199 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.041209 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:41.041327 13927 api.go:53] PageSize not implemented I0713 10:15:41.041299 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.041333 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I0713 10:15:41.041370 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.041379 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.041486 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0713 10:15:41.044299 13927 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0713 10:15:41.044475 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.044570 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.044666 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.044764 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.046838 13927 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0713 10:15:41.047047 13927 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0713 10:15:41.047145 13927 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0713 10:15:41.047185 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.047219 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.047271 13927 dns.go:232] Pre-creating DNS records I0713 10:15:41.047295 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.054435 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:41.057712 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.057813 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.057910 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.128056 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.213796 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.215143 13927 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0713 10:15:41.215368 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.215494 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:41.215536 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:41.215584 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.215774 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:41.215814 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:41.215857 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.216022 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.216167 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.217199 13927 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0713 10:15:41.217311 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0713 10:15:41.217366 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0713 10:15:41.217307 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.217459 13927 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0713 10:15:41.217427 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.217535 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0713 10:15:41.217588 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0713 10:15:41.217542 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.217612 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0713 10:15:41.217647 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.217748 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:41.217777 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.217867 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:41.217897 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.217974 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.218065 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0713 10:15:41.218091 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0713 10:15:41.218200 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:41.218238 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0713 10:15:41.218332 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.218367 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.218457 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:41.218481 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0713 10:15:41.218571 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.218605 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.218687 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.218748 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0713 10:15:41.218846 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:41.218934 13927 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0713 10:15:41.219014 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.219065 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.219118 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.219129 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0713 10:15:41.219121 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.219154 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0713 10:15:41.219178 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } W0713 10:15:41.219161 13927 api.go:53] PageSize not implemented I0713 10:15:41.219201 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0713 10:15:41.219220 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0713 10:15:41.219211 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0713 10:15:41.219271 13927 api.go:53] PageSize not implemented I0713 10:15:41.219283 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.219305 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.219328 13927 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0713 10:15:41.219323 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0713 10:15:41.219484 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.219614 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.219671 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.219724 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.219786 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.219870 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.220024 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0713 10:15:41.220181 13927 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0713 10:15:41.220284 13927 api.go:53] PageSize not implemented I0713 10:15:41.220244 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.220327 13927 dnsname.go:118] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0713 10:15:41.220255 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.220333 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.220379 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:41.220436 13927 api.go:53] PageSize not implemented I0713 10:15:41.220478 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.220624 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.223318 13927 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0713 10:15:41.223384 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.223492 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.223633 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.223740 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.224951 13927 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0713 10:15:41.225008 13927 api.go:53] PageSize not implemented I0713 10:15:41.225022 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.225142 13927 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0713 10:15:41.225217 13927 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0713 10:15:41.227195 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:41.227234 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:41.227259 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:41.227280 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:41.227302 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:41.227323 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:41.227343 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:41.227365 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:41.227388 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:41.227433 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:41.227454 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0713 10:15:41.227484 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:41.227507 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:41.227527 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:41.227548 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:41.227566 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:41.227587 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0713 10:15:41.227609 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:41.227632 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:41.227654 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:41.227675 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:41.227695 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:41.227717 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:41.227735 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.227756 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0713 10:15:41.227777 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0713 10:15:41.227799 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:41.227817 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:41.227836 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:41.227859 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:41.227880 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:41.227900 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:41.228818 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:41.228859 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:41.228871 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:41.228896 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.228936 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0713 10:15:41.228989 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229033 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229060 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229091 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229116 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229148 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0713 10:15:41.229188 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229229 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:41.229249 13927 natgateway.go:117] DescribeNatGateways: { } I0713 10:15:41.229272 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0713 10:15:41.229298 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0713 10:15:41.229333 13927 api.go:53] PageSize not implemented I0713 10:15:41.229343 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0713 10:15:41.229372 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.229387 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.229396 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0713 10:15:41.229420 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:41.229436 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:41.229447 13927 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I0713 10:15:41.229491 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:41.229519 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.229551 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.229569 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.229596 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:41.229624 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:41.229653 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-2","rtb-1"] } I0713 10:15:41.229678 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0713 10:15:41.229697 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-config:lt-1 ok I0713 10:15:41.230111 13927 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok autoscaling-group:bastion.privatekopeio.example.com ok I0713 10:15:41.230107 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.230157 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0713 10:15:41.230201 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0713 10:15:41.230179 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0713 10:15:41.230261 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:41.230233 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0713 10:15:41.230278 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0713 10:15:41.230261 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4 ok I0713 10:15:41.230276 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0713 10:15:41.230301 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } volume:vol-2 ok I0713 10:15:41.230333 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0713 10:15:41.230193 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.230337 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatekopeio.example.com. ok volume:vol-1 ok I0713 10:15:41.230320 13927 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1 ok I0713 10:15:41.230389 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0713 10:15:41.230438 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0713 10:15:41.230461 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:41.230484 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:41.230521 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0713 10:15:41.230642 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:41.230657 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok I0713 10:15:41.230687 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:41.230720 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0713 10:15:41.230855 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:41.230867 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-3 ok I0713 10:15:41.230896 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:41.230977 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.230987 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0713 10:15:41.231010 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0713 10:15:41.231070 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:41.231047 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0713 10:15:41.231103 13927 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0713 10:15:41.231079 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:41.231115 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0713 10:15:41.231140 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0713 10:15:41.231124 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0713 10:15:41.231191 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0713 10:15:41.231177 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0713 10:15:41.231217 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0713 10:15:41.231224 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:41.231249 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0713 10:15:41.231268 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0713 10:15:41.231284 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0713 10:15:41.231327 13927 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0713 10:15:41.231339 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0713 10:15:41.231363 13927 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0713 10:15:41.231425 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0713 10:15:41.231479 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:41.231518 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0713 10:15:41.231584 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:41.231650 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:41.231881 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.38s) === RUN TestLifecycleIPv6 I0713 10:15:41.232031 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:41.232143 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:41.232172 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:41.232204 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:41.232237 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:41.232277 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:41.232327 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:41.232358 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:41.232391 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:41.232417 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:41.232459 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0713 10:15:41.260643 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:41.264001 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.264081 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.335115 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.477367 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.478621 13927 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0713 10:15:41.478888 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0713 10:15:41.478942 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:41.479017 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0713 10:15:41.478984 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.479146 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0713 10:15:41.479156 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.479210 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0713 10:15:41.479346 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:41.479468 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:41.479508 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.479573 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:41.479599 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.479699 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.479751 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:41.479771 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:41.479788 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:41.479811 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.479830 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.479920 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:41.479993 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:41.480018 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:41.480114 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.480818 13927 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0713 10:15:41.480908 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0713 10:15:41.480895 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.480951 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0713 10:15:41.480973 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.481012 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0713 10:15:41.481036 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:41.481122 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0713 10:15:41.481101 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0713 10:15:41.481203 13927 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0713 10:15:41.481201 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0713 10:15:41.481096 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0713 10:15:41.481334 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0713 10:15:41.481343 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.481432 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:41.481473 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.481511 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.481673 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.481749 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.481890 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.481954 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:41.481992 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0713 10:15:41.482153 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0713 10:15:41.482240 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.482335 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.482419 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0713 10:15:41.482353 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0713 10:15:41.482514 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.482613 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:41.482634 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:41.482671 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.482701 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0713 10:15:41.482778 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:41.482802 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:41.482892 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0713 10:15:41.482922 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:41.482980 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:41.483006 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:41.483026 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0713 10:15:41.483105 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:41.483136 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:41.483168 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" } ] } I0713 10:15:41.483219 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0713 10:15:41.483244 13927 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0713 10:15:41.483289 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:41.483311 13927 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0713 10:15:41.483378 13927 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0713 10:15:41.483448 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.483525 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.483589 13927 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0713 10:15:41.483591 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.483636 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", Ipv6CidrBlock: "2001:db8:0:111::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:41.483709 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:41.483736 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:41.483763 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.4.0/22", Ipv6CidrBlock: "2001:db8:0:116::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:41.483800 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.483897 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.484022 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" }] }] } I0713 10:15:41.484147 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.484228 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.484312 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-3" } I0713 10:15:41.484384 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0713 10:15:41.484477 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:41.484600 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0713 10:15:41.484684 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.484726 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.484777 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.484816 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.484898 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.484945 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] }] } I0713 10:15:41.484999 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.484996 13927 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.485050 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0713 10:15:41.485144 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.485196 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.485083 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", Ipv6CidrBlock: "2001:db8:0:114::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "dualstack-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:41.485110 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:113::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "Name", Value: "dualstack-us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:41.486003 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.0.0/22", Ipv6CidrBlock: "2001:db8:0:115::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:41.486071 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.486203 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", Ipv6CidrBlock: "2001:db8:0:112::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I0713 10:15:41.486241 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0713 10:15:41.486386 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:41.486539 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-1" } I0713 10:15:41.486596 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0713 10:15:41.486658 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0713 10:15:41.486831 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.487015 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0713 10:15:41.487183 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.487334 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.487404 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:41.487445 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0713 10:15:41.487634 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0713 10:15:41.487687 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:41.487734 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.487888 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.488048 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.488503 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0713 10:15:41.488826 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0713 10:15:41.488867 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0713 10:15:41.488897 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0713 10:15:41.488926 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0713 10:15:41.488950 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0713 10:15:41.488991 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0713 10:15:41.489039 13927 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0713 10:15:41.489102 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.489144 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0713 10:15:41.489164 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:41.489227 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.489552 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.489698 13927 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-5" },{ SubnetId: "subnet-2" }], Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" }], Type: "network" } I0713 10:15:41.489767 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:41.489807 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I0713 10:15:41.489839 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0713 10:15:41.489865 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0713 10:15:41.489886 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.489969 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-6"] }] } I0713 10:15:41.489994 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-6" } I0713 10:15:41.490015 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.490097 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0713 10:15:41.490121 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0713 10:15:41.490142 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.490184 13927 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.490265 13927 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-2", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.490270 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.490346 13927 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-5"] }] } I0713 10:15:41.490427 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.490441 13927 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-5", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0713 10:15:41.490520 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0713 10:15:41.490567 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0713 10:15:41.490606 13927 tags.go:42] CreateTags { Resources: ["subnet-5"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-2" }] } I0713 10:15:41.490647 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0713 10:15:41.490689 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0713 10:15:41.490726 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0713 10:15:41.490760 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:41.490803 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0713 10:15:41.490848 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0713 10:15:41.490893 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.490865 13927 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0713 10:15:41.490921 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-5"] }] } W0713 10:15:41.490942 13927 api.go:53] PageSize not implemented I0713 10:15:41.490945 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-5" } I0713 10:15:41.490973 13927 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0713 10:15:41.491014 13927 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0713 10:15:41.491049 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.491090 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.491065 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.491109 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.491143 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.491119 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.491208 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.491251 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.491261 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.491273 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.491309 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0713 10:15:41.491361 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.491382 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.491493 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-2", RouteTableId: "rtb-3" } I0713 10:15:41.491537 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.491656 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-2", RouteTableId: "rtb-3" } I0713 10:15:41.491697 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.491804 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0713 10:15:41.493956 13927 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0713 10:15:41.494115 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.494201 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.494255 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.494329 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.496190 13927 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0713 10:15:41.496386 13927 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0713 10:15:41.496478 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:41.496509 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.496532 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.496571 13927 dns.go:232] Pre-creating DNS records I0713 10:15:41.496586 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.500639 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0713 10:15:41.502891 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.502949 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.573550 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.694592 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.695859 13927 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0713 10:15:41.696078 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I0713 10:15:41.696282 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:41.696306 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I0713 10:15:41.696496 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.696595 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.696670 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:41.696695 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:41.696745 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.696778 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:41.697498 13927 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0713 10:15:41.697607 13927 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0713 10:15:41.697614 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.697698 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0713 10:15:41.697703 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0713 10:15:41.697748 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0713 10:15:41.697819 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0713 10:15:41.697851 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0713 10:15:41.697817 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.697928 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0713 10:15:41.698014 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.698054 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.698110 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.698165 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.698243 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.698325 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.698397 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:41.698445 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0713 10:15:41.698521 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:41.698826 13927 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0713 10:15:41.698904 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0713 10:15:41.699015 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] }] } I0713 10:15:41.699115 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:41.699181 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.699290 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.699340 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:41.699365 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0713 10:15:41.699389 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0713 10:15:41.699458 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.699585 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0713 10:15:41.699726 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:41.699791 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.699857 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0713 10:15:41.699892 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0713 10:15:41.699928 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0713 10:15:41.699991 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.700096 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.700186 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0713 10:15:41.700243 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.700292 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.700340 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.700419 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:41.700443 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] }] } I0713 10:15:41.700518 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0713 10:15:41.700561 13927 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0713 10:15:41.700607 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.700644 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.700695 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.700708 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.700733 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.700649 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.700753 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0713 10:15:41.700775 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0713 10:15:41.700795 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0713 10:15:41.700800 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0713 10:15:41.700847 13927 api.go:53] PageSize not implemented I0713 10:15:41.700877 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.700939 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.700988 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:41.701041 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:41.701103 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.701161 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.701210 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0713 10:15:41.701281 13927 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0713 10:15:41.701318 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.701362 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.701336 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.701340 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.701406 13927 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0713 10:15:41.701400 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.701377 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0713 10:15:41.701425 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:41.701452 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0713 10:15:41.701477 13927 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" W0713 10:15:41.701459 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.701496 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.701500 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.701572 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.701704 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:41.701723 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.701760 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:41.701774 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.704285 13927 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0713 10:15:41.704351 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.704456 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.704498 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.704605 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.705922 13927 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0713 10:15:41.706056 13927 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0713 10:15:41.706134 13927 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0713 10:15:41.708036 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:41.708074 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:41.708099 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:41.708119 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-2"] }] } I0713 10:15:41.708142 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0713 10:15:41.708164 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:41.708186 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0713 10:15:41.708207 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:41.708227 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:41.708246 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:41.708267 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:41.708287 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:41.708310 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:41.708336 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0713 10:15:41.708358 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0713 10:15:41.708379 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:41.708400 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:41.708419 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:41.708440 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:41.708459 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:41.708480 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0713 10:15:41.708504 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:41.708525 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:41.708546 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:41.708568 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:41.708589 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:41.708609 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:41.708632 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0713 10:15:41.708654 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0713 10:15:41.708677 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0713 10:15:41.708699 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0713 10:15:41.708721 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:41.708740 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:41.708758 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:41.708780 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:41.708802 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:41.708823 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:41.709953 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:41.709994 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:41.710006 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:41.710033 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710065 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0713 10:15:41.710103 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710142 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710174 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710217 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710248 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710281 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0713 10:15:41.710323 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710370 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:41.710390 13927 natgateway.go:117] DescribeNatGateways: { } I0713 10:15:41.710415 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0713 10:15:41.710440 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0713 10:15:41.710472 13927 api.go:53] PageSize not implemented I0713 10:15:41.710484 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:41.710498 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:41.710507 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0713 10:15:41.710533 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:41.710547 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:41.710561 13927 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0713 10:15:41.710596 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:41.710623 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.710653 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.710670 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.710695 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:41.710721 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:41.710751 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-3","rtb-2"] } I0713 10:15:41.710779 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:41.710800 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0713 10:15:41.710821 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0713 10:15:41.710839 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0713 10:15:41.711225 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:41.711267 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0713 10:15:41.711249 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0713 10:15:41.711299 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0713 10:15:41.711287 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.711310 13927 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0713 10:15:41.711324 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0713 10:15:41.711349 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok iam-instance-profile:nodes.minimal-ipv6.example.com ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0713 10:15:41.711336 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } I0713 10:15:41.711394 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok W0713 10:15:41.711433 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-group:nodes.minimal-ipv6.example.com ok I0713 10:15:41.711443 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:41.711477 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:41.711519 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:41.711542 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok autoscaling-config:lt-1 ok I0713 10:15:41.711582 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-5" } subnet:subnet-5 ok I0713 10:15:41.711376 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I0713 10:15:41.711610 13927 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-2" } nat-gateway:nat-2 ok I0713 10:15:41.711708 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.711772 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } W0713 10:15:41.711956 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:41.711982 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:41.712016 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0713 10:15:41.712068 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0713 10:15:41.712099 13927 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0713 10:15:41.712151 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:41.712202 13927 egressonlyinternetgateways.go:171] DeleteEgressOnlyInternetGateway: { EgressOnlyInternetGatewayId: "eigw-2" } egress-only-internet-gateway:eigw-2 ok autoscaling-config:lt-2 ok I0713 10:15:41.712270 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0713 10:15:41.712371 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0713 10:15:41.712392 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0713 10:15:41.712417 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0713 10:15:41.712444 13927 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com subnet:subnet-1 ok I0713 10:15:41.712460 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0713 10:15:41.712481 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0713 10:15:41.712474 13927 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0713 10:15:41.712501 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0713 10:15:41.712524 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0713 10:15:41.712532 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0713 10:15:41.712529 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-6" } subnet:subnet-6 ok I0713 10:15:41.712575 13927 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-2" } elastic-ip:eipalloc-2 ok I0713 10:15:41.712551 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0713 10:15:41.712617 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0713 10:15:41.712685 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0713 10:15:41.712728 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:41.712782 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:41.713152 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:41.713365 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.48s) === RUN TestLifecycleSharedVPC I0713 10:15:41.713520 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:41.713619 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:41.713648 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:41.713688 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:41.713724 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:41.713767 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:41.713802 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:41.713831 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:41.713869 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:41.713893 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:41.713938 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0713 10:15:41.728418 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:41.730246 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.730313 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.799708 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.868582 13927 iamrole.go:89] ListRoles: { } I0713 10:15:41.869393 13927 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0713 10:15:41.869524 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:41.869681 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0713 10:15:41.869732 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" }] } I0713 10:15:41.869762 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:41.869821 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0713 10:15:41.869859 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0713 10:15:41.869894 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:41.869986 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:41.871629 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:41.871697 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:41.872202 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:41.872403 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.872576 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:41.872731 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:41.872850 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:41.873066 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:41.874555 13927 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0713 10:15:41.874800 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0713 10:15:41.874808 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:41.875335 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0713 10:15:41.875362 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:41.875454 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:41.875440 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0713 10:15:41.875519 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0713 10:15:41.875596 13927 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0713 10:15:41.875558 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0713 10:15:41.875629 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:41.875678 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:41.875738 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:41.875724 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0713 10:15:41.875771 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:41.875816 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0713 10:15:41.875960 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:41.876017 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:41.876050 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:41.876076 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:41.876186 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-12345678" } I0713 10:15:41.876219 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0713 10:15:41.876303 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:41.876336 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:41.877401 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0713 10:15:41.878274 13927 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0713 10:15:41.878398 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.878542 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0713 10:15:41.878624 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.878741 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.878834 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0713 10:15:41.878955 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0713 10:15:41.879047 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.879109 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.879259 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.879417 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.879549 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0713 10:15:41.879618 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.879687 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:41.879758 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.879852 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.880013 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I0713 10:15:41.880111 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.880190 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.880264 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" }] }] } I0713 10:15:41.880340 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:41.880372 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0713 10:15:41.880395 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0713 10:15:41.880474 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0713 10:15:41.880511 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0713 10:15:41.880594 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.881959 13927 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0713 10:15:41.882142 13927 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0713 10:15:41.882231 13927 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0713 10:15:41.882261 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:41.882285 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.882332 13927 dns.go:232] Pre-creating DNS records I0713 10:15:41.882355 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:41.885751 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:41.887475 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.887534 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:41.959189 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.049820 13927 iamrole.go:89] ListRoles: { } I0713 10:15:42.051160 13927 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0713 10:15:42.051363 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.051434 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:42.051539 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.051665 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:42.051747 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:42.051778 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:42.051835 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:42.054769 13927 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0713 10:15:42.054997 13927 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0713 10:15:42.054931 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0713 10:15:42.055177 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0713 10:15:42.055182 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:42.055267 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.055414 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0713 10:15:42.055605 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:42.055739 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.055922 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:42.055971 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:42.060981 13927 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0713 10:15:42.061086 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.061233 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.061340 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.061404 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.061527 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.061606 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.061772 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.063141 13927 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0713 10:15:42.063352 13927 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0713 10:15:42.063476 13927 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0713 10:15:42.065048 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:42.065087 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:42.065106 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:42.065126 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:42.065147 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:42.065166 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:42.065189 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:42.065211 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:42.065235 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:42.065254 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:42.065273 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:42.065294 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:42.065312 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:42.065332 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:42.065351 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:42.065372 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:42.065390 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:42.065409 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:42.065426 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:42.065447 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:42.065468 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:42.066614 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:42.066661 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:42.066673 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:42.066698 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066728 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0713 10:15:42.066760 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066795 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066820 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066844 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066869 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066891 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0713 10:15:42.066917 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066950 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0713 10:15:42.066971 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0713 10:15:42.066998 13927 api.go:53] PageSize not implemented I0713 10:15:42.067011 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.067026 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.067037 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:42.067052 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:42.067063 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:42.067094 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:42.067117 13927 iamrole.go:89] ListRoles: { } I0713 10:15:42.067147 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.067165 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:42.067193 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:42.067216 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:42.067232 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok I0713 10:15:42.067492 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0713 10:15:42.067568 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0713 10:15:42.067474 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-group:nodes.sharedvpc.example.com ok I0713 10:15:42.067620 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedvpc.example.com. ok iam-instance-profile:nodes.sharedvpc.example.com ok I0713 10:15:42.067546 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0713 10:15:42.067675 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:42.067688 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0713 10:15:42.067652 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0713 10:15:42.067726 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0713 10:15:42.067721 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:42.067766 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:42.067800 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0713 10:15:42.067870 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:42.067882 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:42.067901 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:42.067932 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:42.067982 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:42.068014 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0713 10:15:42.068047 13927 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0713 10:15:42.068057 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0713 10:15:42.068076 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0713 10:15:42.068095 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0713 10:15:42.068118 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0713 10:15:42.068125 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0713 10:15:42.068140 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0713 10:15:42.068168 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:42.068327 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.35s) === RUN TestLifecycleComplex I0713 10:15:42.068447 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:42.068531 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:42.068552 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:42.068577 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:42.068600 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:42.068629 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:42.068654 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:42.068671 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:42.068692 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:42.068709 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:42.068739 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0713 10:15:42.089565 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:42.091969 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.092032 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.163143 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.287885 13927 iamrole.go:89] ListRoles: { } I0713 10:15:42.289155 13927 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0713 10:15:42.289345 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:42.289455 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0713 10:15:42.289567 13927 api.go:53] PageSize not implemented I0713 10:15:42.289540 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0713 10:15:42.289631 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0713 10:15:42.289650 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.289718 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:42.289710 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0713 10:15:42.289751 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:42.289902 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:42.289804 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0713 10:15:42.291041 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0713 10:15:42.291357 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0713 10:15:42.289996 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.292207 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.292421 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:42.292685 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:42.292800 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.292986 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:42.293107 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:42.293352 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:42.293406 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:42.293462 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:42.293540 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:42.293775 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.295059 13927 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0713 10:15:42.295612 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.295696 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:42.295697 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0713 10:15:42.295728 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:42.295821 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:42.295792 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:42.295843 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0713 10:15:42.295831 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.295946 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:42.296023 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:42.295957 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0713 10:15:42.296049 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0713 10:15:42.296080 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:42.296048 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.296138 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0713 10:15:42.296229 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.timeout_seconds", Value: "30" },{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:42.296226 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.296307 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0713 10:15:42.296381 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:42.296385 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:42.296490 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:42.296530 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:42.296556 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0713 10:15:42.296626 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:42.296655 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:42.296681 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.296738 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0713 10:15:42.296806 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.296812 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0713 10:15:42.296870 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:42.296896 13927 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0713 10:15:42.296954 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:42.296976 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.297052 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:42.297078 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.297178 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0713 10:15:42.297204 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.297197 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0713 10:15:42.297362 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:42.297366 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" } ] }], VpcId: "vpc-1" } I0713 10:15:42.297474 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0713 10:15:42.297520 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:42.297552 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0713 10:15:42.297639 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:42.297685 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:42.297724 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.297805 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.297868 13927 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0713 10:15:42.297914 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:42.297952 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:42.297982 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0713 10:15:42.298070 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:42.298110 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0713 10:15:42.298183 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:42.298231 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:42.298263 13927 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0713 10:15:42.298300 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:42.298380 13927 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0713 10:15:42.298441 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.298473 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.298518 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.298510 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.298582 13927 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0713 10:15:42.298606 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.298705 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0713 10:15:42.298795 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0713 10:15:42.298960 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0713 10:15:42.299040 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.299119 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0713 10:15:42.299187 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0713 10:15:42.299247 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.299320 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:42.299346 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.299423 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:42.299448 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0713 10:15:42.299513 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0713 10:15:42.299573 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.299631 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0713 10:15:42.299670 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0713 10:15:42.299734 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0713 10:15:42.299840 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.299947 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I0713 10:15:42.300030 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:42.300055 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:42.300079 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0713 10:15:42.300102 13927 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0713 10:15:42.300136 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.300173 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0713 10:15:42.300224 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0713 10:15:42.300226 13927 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:42.300277 13927 api.go:53] PageSize not implemented I0713 10:15:42.300253 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0713 10:15:42.300304 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0713 10:15:42.300385 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0713 10:15:42.300407 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0713 10:15:42.300433 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0713 10:15:42.300517 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0713 10:15:42.300579 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0713 10:15:42.300635 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0713 10:15:42.300722 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.300802 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0713 10:15:42.300854 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0713 10:15:42.300948 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.301029 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0713 10:15:42.301111 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0713 10:15:42.301192 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.301281 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.301366 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" } ] }] } I0713 10:15:42.301451 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" } ] }] } I0713 10:15:42.301533 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0713 10:15:42.301639 13927 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0713 10:15:42.301663 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.301696 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.301708 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.301703 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:42.301837 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:42.301869 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.301926 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.301942 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.304607 13927 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0713 10:15:42.304761 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.304866 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.304958 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.305025 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.306921 13927 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0713 10:15:42.307112 13927 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0713 10:15:42.307199 13927 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0713 10:15:42.307237 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.307269 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:42.307321 13927 dns.go:232] Pre-creating DNS records I0713 10:15:42.307344 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:42.314739 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:42.318496 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.318584 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.392221 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.566531 13927 iamrole.go:89] ListRoles: { } I0713 10:15:42.567826 13927 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0713 10:15:42.568023 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:42.568185 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0713 10:15:42.568260 13927 api.go:53] PageSize not implemented I0713 10:15:42.568222 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:42.568305 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.568417 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.568524 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.568587 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:42.568606 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:42.568639 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.568666 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:42.568760 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.568869 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:42.571198 13927 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0713 10:15:42.571412 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.571610 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:42.571610 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:42.571721 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:42.571774 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:42.571564 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:42.571889 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:42.571935 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:42.572077 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:42.572039 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.572221 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:42.572289 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:42.572326 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.572516 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:42.573125 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.573643 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:42.573805 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.574419 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:42.574695 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.574921 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.575075 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.575247 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.575338 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.575429 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:42.575466 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:42.575497 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:42.575566 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:42.575709 13927 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0713 10:15:42.575876 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.575963 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.576001 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.576077 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.576095 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.576109 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.576138 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.576176 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:42.576216 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:42.576250 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:42.576253 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.576281 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:42.576367 13927 api.go:53] PageSize not implemented I0713 10:15:42.576462 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.576680 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.576887 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.577016 13927 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0713 10:15:42.577054 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.577109 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.577101 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:42.577130 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.577198 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.577246 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.577178 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:42.577262 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.577401 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:42.577429 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.581936 13927 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0713 10:15:42.582068 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.582242 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.582345 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.582463 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.584141 13927 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0713 10:15:42.584330 13927 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0713 10:15:42.584434 13927 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0713 10:15:42.588425 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.590421 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.597223 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:42.599544 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.599603 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.678055 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.802030 13927 iamrole.go:89] ListRoles: { } I0713 10:15:42.803305 13927 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0713 10:15:42.803554 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.803690 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.803733 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:42.803778 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.803845 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W0713 10:15:42.803862 13927 api.go:53] PageSize not implemented I0713 10:15:42.803879 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:42.803913 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:42.804015 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:42.804072 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.804164 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:42.804225 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.804249 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:42.805464 13927 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0713 10:15:42.805572 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.805666 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:42.805609 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:42.805760 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:42.805796 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:42.805709 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.805832 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:42.805849 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:42.805901 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:42.805903 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:42.805921 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:42.805937 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.806028 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:42.806057 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:42.806207 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:42.806261 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.806347 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.806406 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.806450 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.806499 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:42.806582 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:42.806625 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:42.806707 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:42.806732 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:42.806783 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.806857 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:42.806936 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:42.806987 13927 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0713 10:15:42.807104 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.807174 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.807234 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.807255 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.807306 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.807320 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.807318 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:42.807349 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.807370 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:42.807393 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:42.807415 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:42.807434 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:42.807381 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.807545 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0713 10:15:42.807565 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:42.807582 13927 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0713 10:15:42.807601 13927 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0713 10:15:42.807622 13927 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0713 10:15:42.807673 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.807678 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0713 10:15:42.807747 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.807777 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } W0713 10:15:42.807801 13927 api.go:53] PageSize not implemented I0713 10:15:42.807910 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:42.807980 13927 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0713 10:15:42.808004 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.808038 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.808038 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:42.808053 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.808085 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:42.808099 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:42.808113 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.808153 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:42.808184 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:42.808193 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:42.813786 13927 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0713 10:15:42.814194 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.814323 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.814382 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.814512 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.816595 13927 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0713 10:15:42.816726 13927 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0713 10:15:42.816812 13927 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0713 10:15:42.816841 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:42.816862 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:42.816896 13927 dns.go:232] Pre-creating DNS records I0713 10:15:42.821344 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:42.824034 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.824130 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:42.894783 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.018251 13927 iamrole.go:89] ListRoles: { } I0713 10:15:43.019560 13927 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0713 10:15:43.019797 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0713 10:15:43.019912 13927 api.go:53] PageSize not implemented I0713 10:15:43.019935 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.019963 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:43.020045 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:43.020078 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:43.020133 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.020182 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.020323 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:43.020414 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.020528 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:43.020613 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0713 10:15:43.020713 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:43.021487 13927 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0713 10:15:43.021572 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.021661 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:43.021686 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:43.021674 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:43.021691 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.021731 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.021797 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:43.021796 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.021857 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:43.021881 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.021908 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.021930 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:43.021942 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:43.022041 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:43.022072 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.022122 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.022166 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.022250 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:43.022275 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:43.022375 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:43.022408 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:43.022434 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.022517 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:43.022612 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:43.022653 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.022714 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.022800 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:43.022846 13927 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0713 10:15:43.022959 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.022980 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.023046 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.023063 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.023102 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.023098 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.023133 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:43.023162 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.023183 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.023200 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:43.023252 13927 api.go:53] PageSize not implemented I0713 10:15:43.023359 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.023501 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.023601 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.023672 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.023721 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.023772 13927 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0713 10:15:43.023812 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:43.023857 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:43.023828 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.023886 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.023896 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.024002 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.024039 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.024056 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.024130 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:43.024158 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.027082 13927 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0713 10:15:43.027151 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.027292 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.027348 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.027498 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.029564 13927 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0713 10:15:43.029767 13927 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0713 10:15:43.029859 13927 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0713 10:15:43.031944 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.033980 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.040895 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:43.043271 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.043332 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.114098 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.260697 13927 iamrole.go:89] ListRoles: { } I0713 10:15:43.262018 13927 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0713 10:15:43.262333 13927 api.go:53] PageSize not implemented I0713 10:15:43.262429 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.262498 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:43.262577 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:43.264611 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:43.264719 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:43.264819 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0713 10:15:43.264920 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:43.265016 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.265233 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.265449 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:43.265536 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.265770 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:43.265859 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.267002 13927 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0713 10:15:43.267122 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.267183 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:43.267276 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.267322 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:43.267252 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:43.267202 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.267796 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:43.267876 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:43.267828 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:43.267854 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.267911 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.267949 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:43.267933 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.268048 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:43.268193 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:43.268250 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.268371 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.268495 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:43.268531 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.268614 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.268698 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.268809 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.268883 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:43.269041 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:43.269077 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:43.269183 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:43.269268 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:43.269337 13927 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0713 10:15:43.269422 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.269486 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.269500 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.269574 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.269588 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.269616 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.269647 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:43.269673 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.269696 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.269721 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:43.269787 13927 api.go:53] PageSize not implemented I0713 10:15:43.269900 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.269994 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.270115 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.270260 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.270564 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.270971 13927 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0713 10:15:43.271031 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.271118 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.271078 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:43.271149 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:43.271146 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.271315 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:43.271351 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.271401 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.271438 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.271448 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.276376 13927 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0713 10:15:43.276531 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.276649 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.276694 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.276868 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.280022 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.280086 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.280238 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:43.280435 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:43.280537 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:43.280578 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.280609 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:43.280664 13927 dns.go:232] Pre-creating DNS records I0713 10:15:43.289271 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:43.293338 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.293431 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.365155 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.523155 13927 iamrole.go:89] ListRoles: { } I0713 10:15:43.524496 13927 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0713 10:15:43.524719 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0713 10:15:43.524870 13927 api.go:53] PageSize not implemented I0713 10:15:43.524844 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:43.524881 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.525019 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:43.525076 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:43.525191 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.525302 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:43.525332 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:43.525393 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.525433 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0713 10:15:43.525540 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0713 10:15:43.525683 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.525821 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:43.526395 13927 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0713 10:15:43.526488 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.526611 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:43.526568 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.526655 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:43.526657 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.526714 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:43.526769 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.526746 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.526792 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:43.526856 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:43.526882 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.526883 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:43.526930 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.526933 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.526953 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:43.526984 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:43.527091 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:43.527125 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.527205 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:43.527227 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:43.527309 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:43.527350 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:43.527435 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:43.527460 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.527507 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.527576 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.527619 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:43.527662 13927 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0713 10:15:43.527771 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.527845 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.527883 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.527893 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.527865 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.527918 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.527938 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:43.527961 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.527980 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.527998 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:43.528006 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:43.528048 13927 api.go:53] PageSize not implemented I0713 10:15:43.528135 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.528236 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.528441 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.528606 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.528938 13927 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0713 10:15:43.529006 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.529063 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.529076 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.529004 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:43.529161 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:43.529182 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:43.529198 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.529243 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.529277 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.529286 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.531872 13927 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0713 10:15:43.531959 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.532073 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.532119 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.532231 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.536113 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:43.536695 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:43.536817 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0713 10:15:43.540312 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.542257 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.542324 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.547669 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:43.550809 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.550882 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.550921 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.622231 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.776789 13927 iamrole.go:89] ListRoles: { } I0713 10:15:43.778086 13927 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0713 10:15:43.778288 13927 api.go:53] PageSize not implemented I0713 10:15:43.778402 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.778543 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.778659 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:43.778699 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:43.778789 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.778881 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:43.778939 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0713 10:15:43.779007 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0713 10:15:43.779071 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:43.779089 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:43.779122 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.779153 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:43.779243 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:43.780187 13927 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0713 10:15:43.780283 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.780406 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:43.780388 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.780481 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:43.780477 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.780519 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:43.780523 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:43.780518 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.780581 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:43.780639 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.780658 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:43.780584 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.780716 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0713 10:15:43.780794 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.780878 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:43.780907 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:43.781020 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:43.781052 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:43.781134 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:43.781167 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.781252 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:43.781275 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:43.781348 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:43.781369 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.781415 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:43.781458 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:43.781541 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:43.781639 13927 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0713 10:15:43.781709 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.781798 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.781874 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0713 10:15:43.781876 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0713 10:15:43.781918 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.781953 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.781952 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.781985 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.782005 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:43.782005 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.782029 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:43.782048 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:43.782065 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:43.782117 13927 api.go:53] PageSize not implemented I0713 10:15:43.782364 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:43.782452 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:43.782581 13927 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0713 10:15:43.782602 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.782646 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.782644 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:43.782660 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.782694 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:43.782708 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:43.782722 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.782752 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:43.782781 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:43.782790 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:43.785447 13927 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0713 10:15:43.785526 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.785651 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.785725 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.785840 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.787208 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:43.787406 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:43.787499 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:43.787531 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:43.787557 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:43.787592 13927 dns.go:232] Pre-creating DNS records I0713 10:15:43.792025 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:43.794394 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.794450 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.794484 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:43.866339 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.032467 13927 iamrole.go:89] ListRoles: { } I0713 10:15:44.033828 13927 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0713 10:15:44.034067 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0713 10:15:44.034169 13927 api.go:53] PageSize not implemented I0713 10:15:44.034120 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:44.034263 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0713 10:15:44.034386 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:44.034480 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0713 10:15:44.034538 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.036412 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.036583 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.036711 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:44.036747 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:44.036902 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:44.037139 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:44.037323 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.040742 13927 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0713 10:15:44.040893 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.040965 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:44.041031 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:44.041060 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:44.041080 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:44.041021 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:44.041182 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:44.041164 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:44.041297 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:44.041296 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:44.041359 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:44.041418 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:44.041455 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.041508 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:44.041593 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:44.041643 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.041740 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:44.041766 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.041847 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.041930 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.041979 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.042055 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:44.042078 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.042150 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:44.042174 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.042246 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.042290 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:44.042711 13927 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0713 10:15:44.042934 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.043051 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.043035 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.043141 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.043169 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.043220 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.043262 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:44.043285 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.043318 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:44.043357 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:44.043404 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:44.043396 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0713 10:15:44.043491 13927 api.go:53] PageSize not implemented I0713 10:15:44.043621 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.043779 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.043928 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.044037 13927 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0713 10:15:44.044073 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.044120 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.044128 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:44.044142 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.044194 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:44.044212 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:44.044242 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.044319 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.044348 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.044452 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.048692 13927 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0713 10:15:44.048808 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.048927 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.048967 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.049159 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.050722 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:44.050923 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:44.051027 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0713 10:15:44.052841 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.054789 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.054838 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.054921 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.060490 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:44.062866 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.062927 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.062961 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.062990 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.133410 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.276857 13927 iamrole.go:89] ListRoles: { } I0713 10:15:44.278746 13927 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0713 10:15:44.278966 13927 api.go:53] PageSize not implemented I0713 10:15:44.278929 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0713 10:15:44.279753 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:44.279899 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:44.280018 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:44.280088 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.280149 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:44.280245 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:44.280284 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:44.280348 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0713 10:15:44.280485 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.280632 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:44.280726 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.280845 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.281639 13927 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0713 10:15:44.281807 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:44.281886 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:44.281926 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:44.282030 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.282155 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0713 10:15:44.282250 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:44.282322 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:44.282257 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.282324 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:44.282413 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:44.282409 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:44.282466 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:44.282455 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:44.282610 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:44.282658 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.282750 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.282869 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:44.282897 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.282997 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.283065 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.283122 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:44.283225 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:44.283257 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:44.283365 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:44.283406 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.283540 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:44.283572 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.283685 13927 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0713 10:15:44.283781 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.283882 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.283900 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.283978 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.283996 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.284042 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.284038 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.284086 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:44.284127 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:44.284163 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:44.284197 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:44.284290 13927 api.go:53] PageSize not implemented I0713 10:15:44.284413 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.284590 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.284679 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.285093 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.285288 13927 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0713 10:15:44.285328 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.285385 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.285386 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:44.285410 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.285469 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:44.285496 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:44.285523 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.285565 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.285629 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.285661 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.291057 13927 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0713 10:15:44.291294 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.291456 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.291918 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.292177 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.294741 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:44.295017 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:44.295124 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:44.295172 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.295221 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:44.295284 13927 dns.go:232] Pre-creating DNS records I0713 10:15:44.302593 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0713 10:15:44.306810 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.306916 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.306978 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.307041 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.379779 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.560521 13927 iamrole.go:89] ListRoles: { } I0713 10:15:44.562698 13927 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0713 10:15:44.563084 13927 api.go:53] PageSize not implemented I0713 10:15:44.565416 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0713 10:15:44.565682 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:44.566111 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0713 10:15:44.566391 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.566517 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:44.566551 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:44.566610 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.567618 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0713 10:15:44.567796 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0713 10:15:44.567933 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.568123 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.568459 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0713 10:15:44.568660 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.570329 13927 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0713 10:15:44.570506 13927 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:44.570668 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:44.570453 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.570755 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0713 10:15:44.570830 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:44.570864 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0713 10:15:44.570877 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:44.570927 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0713 10:15:44.570986 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:44.570921 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0713 10:15:44.571017 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0713 10:15:44.571076 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:44.571145 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.571285 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.571376 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.571499 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0713 10:15:44.571579 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.571806 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.571902 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.572027 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.572168 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:44.572208 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.572293 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0713 10:15:44.572468 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:44.572514 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0713 10:15:44.572666 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:44.572797 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.572964 13927 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0713 10:15:44.573253 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.573363 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.573704 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.573917 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.574121 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:44.574413 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.574632 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.574701 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.574719 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.574758 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.574792 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0713 10:15:44.574830 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0713 10:15:44.574863 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0713 10:15:44.574893 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0713 10:15:44.574979 13927 api.go:53] PageSize not implemented I0713 10:15:44.575352 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.575686 13927 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0713 10:15:44.575749 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:44.575822 13927 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0713 10:15:44.575845 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0713 10:15:44.575868 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.575922 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.575979 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.575997 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.581028 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.581261 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.581328 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.583120 13927 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0713 10:15:44.583483 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.583670 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.585470 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.585666 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.589007 13927 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0713 10:15:44.589319 13927 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0713 10:15:44.589422 13927 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0713 10:15:44.589586 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:44.589651 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:44.589696 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:44.589731 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:44.589766 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:44.589800 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:44.589832 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:44.589868 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:44.589907 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:44.589946 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:44.589981 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:44.590012 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:44.590050 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:44.590082 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:44.590119 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:44.590157 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:44.590191 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:44.590227 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:44.590273 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:44.590309 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:44.590347 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0713 10:15:44.590384 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:44.590418 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:44.590454 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:44.590492 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:44.590530 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:44.590575 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:44.593346 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:44.593422 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:44.593439 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:44.593476 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593542 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0713 10:15:44.593604 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593662 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593712 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593765 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593806 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593855 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0713 10:15:44.593909 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.593978 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0713 10:15:44.594018 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0713 10:15:44.594069 13927 api.go:53] PageSize not implemented I0713 10:15:44.594088 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:44.594112 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:44.594127 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0713 10:15:44.594166 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:44.594233 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:44.594255 13927 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0713 10:15:44.594327 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:44.594371 13927 iamrole.go:89] ListRoles: { } I0713 10:15:44.594422 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.594449 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:44.594491 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:44.594538 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:44.594579 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0713 10:15:44.595196 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0713 10:15:44.595378 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0713 10:15:44.595457 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0713 10:15:44.595543 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0713 10:15:44.595614 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0713 10:15:44.595775 13927 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok autoscaling-group:nodes.complex.example.com ok I0713 10:15:44.595932 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I0713 10:15:44.596208 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I0713 10:15:44.596330 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:44.596434 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:44.596526 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0713 10:15:44.596611 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0713 10:15:44.596691 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0713 10:15:44.596789 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0713 10:15:44.596941 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:44.597027 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 } ] } W0713 10:15:44.597279 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:44.597319 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:44.597394 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:44.597474 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0713 10:15:44.597553 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:44.597623 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0713 10:15:44.598008 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:44.598044 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:44.598159 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0713 10:15:44.598261 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:44.598357 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0713 10:15:44.598426 13927 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0713 10:15:44.598463 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0713 10:15:44.598518 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0713 10:15:44.598594 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0713 10:15:44.598657 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0713 10:15:44.598692 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0713 10:15:44.598743 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0713 10:15:44.598835 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:44.598950 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:44.599097 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:44.609402 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.54s) === RUN TestLifecycleExternalLB I0713 10:15:44.609798 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:44.609976 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:44.610048 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:44.610124 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:44.610200 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:44.610273 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:44.610345 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:44.610400 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:44.610467 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:44.610518 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:44.610594 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0713 10:15:44.634163 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:44.637151 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.637246 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.708761 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.826062 13927 iamrole.go:89] ListRoles: { } I0713 10:15:44.827693 13927 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0713 10:15:44.827972 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:44.828171 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.828261 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0713 10:15:44.828346 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0713 10:15:44.828376 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } I0713 10:15:44.830026 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.830203 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:44.830349 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:44.830419 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0713 10:15:44.830531 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } W0713 10:15:44.832444 13927 api.go:53] PageSize not implemented W0713 10:15:44.832525 13927 api.go:53] PageSize not implemented I0713 10:15:44.832505 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0713 10:15:44.832526 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0713 10:15:44.832603 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0713 10:15:44.832638 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } I0713 10:15:44.832612 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0713 10:15:44.833126 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.833286 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0713 10:15:44.833389 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } W0713 10:15:44.833441 13927 api.go:53] PageSize not implemented I0713 10:15:44.833533 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:44.833601 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:44.833629 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0713 10:15:44.833675 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:44.833690 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.833805 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.833980 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.834088 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:44.834126 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:44.834156 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:44.834210 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.834248 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:44.834395 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:44.835619 13927 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0713 10:15:44.835825 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.835926 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:44.836049 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0713 10:15:44.836139 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:44.836185 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] } I0713 10:15:44.836352 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0713 10:15:44.836466 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0713 10:15:44.836651 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }], VpcId: "vpc-1" } I0713 10:15:44.836732 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:44.836775 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:44.836837 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0713 10:15:44.836906 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.837077 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:44.837214 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:44.837260 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:44.837333 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:44.837494 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.837572 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:44.837686 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0713 10:15:44.837786 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.838246 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:44.838314 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:44.838429 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:44.855315 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0713 10:15:44.855474 13927 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0713 10:15:44.855652 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:44.855777 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:44.855851 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:44.855812 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0713 10:15:44.860958 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0713 10:15:44.861366 13927 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0713 10:15:44.861562 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.861824 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.862063 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.862270 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.862421 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.862740 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.862911 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.863045 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.863191 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:44.863245 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.863328 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:44.863412 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.863569 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.863676 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.863822 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.863947 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.864049 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.864101 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.864265 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:44.864314 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0713 10:15:44.864360 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:44.864412 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.864566 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0713 10:15:44.864852 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" }] }] } I0713 10:15:44.865078 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0713 10:15:44.866068 13927 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0713 10:15:44.866275 13927 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0713 10:15:44.866401 13927 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0713 10:15:44.866454 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:44.866499 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:44.866579 13927 dns.go:232] Pre-creating DNS records I0713 10:15:44.866617 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:44.871389 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:44.873276 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.873349 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:44.943923 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.040212 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.041133 13927 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0713 10:15:45.041328 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0713 10:15:45.041411 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0713 10:15:45.041447 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } I0713 10:15:45.041408 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0713 10:15:45.041521 13927 api.go:53] PageSize not implemented I0713 10:15:45.041484 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0713 10:15:45.041569 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0713 10:15:45.041595 13927 api.go:53] PageSize not implemented I0713 10:15:45.041603 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } I0713 10:15:45.041568 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:45.041559 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0713 10:15:45.041654 13927 api.go:53] PageSize not implemented I0713 10:15:45.041697 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:45.041682 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:45.041733 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:45.041745 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:45.041763 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0713 10:15:45.041776 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.041925 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.042008 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.042087 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:45.043403 13927 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0713 10:15:45.043503 13927 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0713 10:15:45.043552 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0713 10:15:45.043513 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.043650 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.043771 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0713 10:15:45.043924 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.043968 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0713 10:15:45.044135 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:45.044183 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:45.044273 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.044413 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:45.044450 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:45.046244 13927 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0713 10:15:45.046353 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.046429 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:45.046573 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.046698 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.046885 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:45.046975 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:45.047135 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.048819 13927 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0713 10:15:45.049052 13927 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0713 10:15:45.049138 13927 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0713 10:15:45.050884 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:45.050926 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:45.050949 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:45.050968 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:45.050985 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:45.051005 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:45.051022 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:45.051042 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:45.051061 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:45.051082 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:45.051105 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:45.051124 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:45.051140 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:45.051160 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:45.051178 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:45.051198 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:45.051217 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:45.051241 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:45.051259 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:45.051276 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:45.051294 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:45.051314 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:45.051334 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:45.051353 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.052084 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:45.052120 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:45.052133 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:45.052158 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052187 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0713 10:15:45.052220 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052256 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052291 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052358 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052402 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052438 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0713 10:15:45.052500 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052548 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0713 10:15:45.052584 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0713 10:15:45.052630 13927 api.go:53] PageSize not implemented I0713 10:15:45.052650 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:45.052670 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:45.052686 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:45.052707 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:45.052722 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0713 10:15:45.052763 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:45.052800 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.052840 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.052867 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.052902 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:45.052938 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:45.052966 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.externallb.example.com ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0713 10:15:45.053275 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0713 10:15:45.053290 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0713 10:15:45.053337 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } keypair:key-1 ok iam-instance-profile:masters.externallb.example.com ok autoscaling-config:lt-1 ok I0713 10:15:45.053359 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0713 10:15:45.053393 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0713 10:15:45.053393 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0713 10:15:45.053468 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0713 10:15:45.053527 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:45.053362 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0713 10:15:45.053590 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.053695 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0713 10:15:45.053927 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.053953 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:45.053990 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:45.054048 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0713 10:15:45.054164 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.054184 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:45.054222 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:45.054316 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:45.054333 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0713 10:15:45.054406 13927 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0713 10:15:45.054423 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0713 10:15:45.054459 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0713 10:15:45.054500 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0713 10:15:45.054548 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0713 10:15:45.054563 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0713 10:15:45.054595 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0713 10:15:45.054657 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:45.054743 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:45.054821 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:45.055081 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.45s) === RUN TestLifecycleSharedSubnet I0713 10:15:45.055253 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:45.055372 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:45.055406 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:45.055446 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:45.055485 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:45.055535 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:45.055578 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:45.055609 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:45.055659 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:45.055689 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:45.055738 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0713 10:15:45.084482 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:45.088561 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.088650 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.162851 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.241948 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.243124 13927 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0713 10:15:45.243358 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.243491 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:45.243581 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:45.243659 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.243776 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0713 10:15:45.243809 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:45.243900 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:45.243866 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0713 10:15:45.243934 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" }] }] } I0713 10:15:45.243944 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.244041 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:45.244033 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0713 10:15:45.244114 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:45.244145 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:45.244190 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:45.244206 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:45.245547 13927 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0713 10:15:45.245661 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0713 10:15:45.245715 13927 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0713 10:15:45.245728 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0713 10:15:45.245817 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0713 10:15:45.245902 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.245917 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0713 10:15:45.246078 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:45.246148 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:45.246180 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:45.246204 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0713 10:15:45.246333 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.246336 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.246457 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:45.246560 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:45.246617 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.246660 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:45.246706 13927 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] } I0713 10:15:45.247967 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0713 10:15:45.250054 13927 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0713 10:15:45.250268 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0713 10:15:45.250429 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.250525 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.250682 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0713 10:15:45.250931 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.251087 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.251316 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.251523 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0713 10:15:45.251684 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0713 10:15:45.251824 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.251910 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.252006 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.252179 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0713 10:15:45.252335 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.252477 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.252605 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" }] }] } I0713 10:15:45.252744 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.252955 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0713 10:15:45.253261 13927 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0713 10:15:45.253495 13927 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0713 10:15:45.253621 13927 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0713 10:15:45.253698 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.253764 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.253858 13927 dns.go:232] Pre-creating DNS records I0713 10:15:45.253891 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.259854 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:45.263985 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.264135 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.334777 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.428439 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.429716 13927 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0713 10:15:45.429870 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.430022 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:45.430130 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:45.430117 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:45.430166 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:45.430212 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:45.430329 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.432080 13927 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0713 10:15:45.432262 13927 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0713 10:15:45.432174 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0713 10:15:45.432366 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.432458 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0713 10:15:45.432421 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0713 10:15:45.432542 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:45.432593 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.432660 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0713 10:15:45.440650 13927 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0713 10:15:45.440873 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.441050 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.441126 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.441258 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.442602 13927 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0713 10:15:45.442750 13927 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0713 10:15:45.442873 13927 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0713 10:15:45.444401 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:45.444440 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:45.444460 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:45.444480 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:45.444500 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:45.444520 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:45.444541 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:45.444565 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:45.444587 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:45.444606 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:45.444623 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:45.444641 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:45.444660 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:45.444680 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:45.444700 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:45.444718 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:45.444735 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:45.444755 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:45.444776 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.445563 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:45.445605 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:45.445617 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:45.445665 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445706 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0713 10:15:45.445743 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445778 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445802 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445828 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445853 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445873 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0713 10:15:45.445896 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445922 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0713 10:15:45.445943 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0713 10:15:45.445969 13927 api.go:53] PageSize not implemented I0713 10:15:45.445982 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:45.445998 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:45.446009 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:45.446023 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:45.446035 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:45.446070 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:45.446096 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.446127 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.446145 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.446174 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:45.446197 13927 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0713 10:15:45.446428 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.sharedsubnet.example.com ok I0713 10:15:45.446476 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.446471 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0713 10:15:45.446546 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.446545 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.446557 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } iam-instance-profile:nodes.sharedsubnet.example.com ok security-group:sg-1 ok I0713 10:15:45.446578 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:45.446590 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } autoscaling-config:lt-1 ok I0713 10:15:45.446636 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0713 10:15:45.446498 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedsubnet.example.com. ok I0713 10:15:45.446650 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0713 10:15:45.446724 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.446769 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0713 10:15:45.446893 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.446908 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:45.446930 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:45.446985 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.447015 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0713 10:15:45.447028 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0713 10:15:45.447080 13927 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0713 10:15:45.447090 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0713 10:15:45.447119 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0713 10:15:45.447136 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0713 10:15:45.447153 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0713 10:15:45.447328 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedSubnet I0713 10:15:45.447467 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:45.447556 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:45.447578 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:45.447603 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:45.447626 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:45.447658 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:45.447683 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:45.447700 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:45.447723 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:45.447740 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:45.447767 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0713 10:15:45.468416 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:45.470385 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.470442 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.470477 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.540851 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.634034 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.636062 13927 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0713 10:15:45.636369 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0713 10:15:45.636684 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0713 10:15:45.636625 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:45.636772 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0713 10:15:45.636803 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:45.636905 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0713 10:15:45.636887 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:45.636964 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0713 10:15:45.637035 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:45.637083 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0713 10:15:45.637144 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0713 10:15:45.637056 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.637296 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:45.637374 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:45.637404 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:45.637447 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] }] } I0713 10:15:45.637605 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:45.637734 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:45.638935 13927 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0713 10:15:45.639059 13927 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0713 10:15:45.639104 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0713 10:15:45.639136 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0713 10:15:45.639048 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0713 10:15:45.639140 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0713 10:15:45.639219 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0713 10:15:45.639268 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-su-5ol32q", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-su-5ol32q" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0713 10:15:45.639398 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.timeout_seconds", Value: "30" },{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0713 10:15:45.639382 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0713 10:15:45.639453 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0713 10:15:45.639612 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0713 10:15:45.639599 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0713 10:15:45.639706 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0713 10:15:45.639771 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0713 10:15:45.639827 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0713 10:15:45.640039 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:45.640160 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:45.640213 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:45.640256 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0713 10:15:45.640370 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.640484 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0713 10:15:45.640576 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:45.640670 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0713 10:15:45.640777 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:45.640822 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:45.640873 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0713 10:15:45.640967 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:45.641018 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:45.641067 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:45.641118 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.641153 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:45.641199 13927 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" } ] } I0713 10:15:45.641279 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:45.641343 13927 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0713 10:15:45.641495 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0713 10:15:45.641922 13927 executor.go:111] Tasks: 67 done / 107 total; 27 can run W0713 10:15:45.642041 13927 api.go:53] PageSize not implemented I0713 10:15:45.642135 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:45.642210 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:45.642088 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0713 10:15:45.642185 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.642271 13927 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" }] } I0713 10:15:45.642357 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.642336 13927 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-su-5ol32q", SubnetMappings: [{ SubnetId: "subnet-abcdef" }], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }], Type: "network" } I0713 10:15:45.642396 13927 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0713 10:15:45.642339 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.642452 13927 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0713 10:15:45.642530 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0713 10:15:45.642734 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.642975 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0713 10:15:45.643079 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" }] }] } I0713 10:15:45.643379 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:45.643545 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.643797 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.643907 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.644011 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.644242 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0713 10:15:45.644409 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I0713 10:15:45.644562 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0713 10:15:45.644643 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1", Port: 22, Protocol: "TCP" } I0713 10:15:45.644607 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } W0713 10:15:45.644742 13927 api.go:53] PageSize not implemented I0713 10:15:45.644760 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.644749 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" }] }] } I0713 10:15:45.644903 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.645065 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:45.645169 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0713 10:15:45.645306 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.645456 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.645600 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.645766 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.645910 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" }] }] } I0713 10:15:45.646053 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0713 10:15:45.646195 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0713 10:15:45.646331 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:45.646478 13927 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0713 10:15:45.646520 13927 api.go:53] PageSize not implemented I0713 10:15:45.646567 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.646572 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0713 10:15:45.646740 13927 api.go:53] PageSize not implemented I0713 10:15:45.646710 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:45.646803 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.653151 13927 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0713 10:15:45.653502 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.654269 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.655050 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.655144 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.657573 13927 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0713 10:15:45.657896 13927 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0713 10:15:45.658029 13927 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0713 10:15:45.658080 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.658122 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.658186 13927 dns.go:232] Pre-creating DNS records I0713 10:15:45.658212 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.664862 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:45.668666 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.668757 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.668824 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.738878 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.812953 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.814197 13927 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0713 10:15:45.814457 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:45.814557 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:45.814583 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:45.814614 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:45.814715 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.814715 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:45.814813 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.816627 13927 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0713 10:15:45.816776 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0713 10:15:45.816755 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0713 10:15:45.816821 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0713 10:15:45.816725 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0713 10:15:45.816833 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0713 10:15:45.816870 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0713 10:15:45.816853 13927 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0713 10:15:45.816885 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0713 10:15:45.817038 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.817095 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0713 10:15:45.817238 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:45.817274 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0713 10:15:45.817390 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0713 10:15:45.817529 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:45.817581 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.817689 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:45.817759 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0713 10:15:45.818597 13927 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0713 10:15:45.818744 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } W0713 10:15:45.818810 13927 api.go:53] PageSize not implemented I0713 10:15:45.818784 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:45.818855 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:45.818844 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.818875 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0713 10:15:45.818856 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.818921 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.818954 13927 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0713 10:15:45.818915 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0713 10:15:45.819018 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } I0713 10:15:45.819070 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0713 10:15:45.819098 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } W0713 10:15:45.819156 13927 api.go:53] PageSize not implemented I0713 10:15:45.819169 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.819400 13927 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0713 10:15:45.819444 13927 api.go:53] PageSize not implemented I0713 10:15:45.819473 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:45.819529 13927 dnsname.go:118] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0713 10:15:45.819490 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.819642 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0713 10:15:45.819709 13927 api.go:53] PageSize not implemented I0713 10:15:45.819744 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.822515 13927 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0713 10:15:45.822584 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.822676 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.822752 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.822917 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.825179 13927 executor.go:111] Tasks: 103 done / 107 total; 2 can run W0713 10:15:45.825271 13927 api.go:53] PageSize not implemented I0713 10:15:45.825288 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.825420 13927 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0713 10:15:45.825527 13927 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0713 10:15:45.828741 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:45.828799 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:45.828838 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:45.828874 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:45.828907 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:45.828941 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:45.828989 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:45.829035 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:45.829077 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0713 10:15:45.829120 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:45.829153 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:45.829194 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:45.829231 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:45.829267 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:45.829301 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:45.829336 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:45.829375 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:45.829412 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:45.829449 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:45.829481 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:45.829519 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:45.829554 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:45.831035 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:45.831096 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:45.831113 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:45.831150 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831208 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0713 10:15:45.831275 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831336 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831384 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831429 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831470 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831504 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0713 10:15:45.831547 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831592 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0713 10:15:45.831627 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0713 10:15:45.831668 13927 api.go:53] PageSize not implemented I0713 10:15:45.831683 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0713 10:15:45.831722 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:45.831745 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:45.831760 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0713 10:15:45.831797 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:45.831818 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:45.831835 13927 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" ] } I0713 10:15:45.831899 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:45.831946 13927 iamrole.go:89] ListRoles: { } I0713 10:15:45.831991 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:45.832021 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:45.832061 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:45.832098 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:45.832643 13927 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1 ok I0713 10:15:45.832653 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:bastion.private-shared-subnet.example.com ok I0713 10:15:45.832735 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0713 10:15:45.832748 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4 ok I0713 10:15:45.832792 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } autoscaling-group:nodes.private-shared-subnet.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0713 10:15:45.832798 13927 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok iam-instance-profile:nodes.private-shared-subnet.example.com ok I0713 10:15:45.832819 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0713 10:15:45.832749 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0713 10:15:45.832896 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.832869 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0713 10:15:45.832915 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:45.832712 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0713 10:15:45.832934 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0713 10:15:45.832951 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:45.832981 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0713 10:15:45.833000 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:45.833049 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:45.833099 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0713 10:15:45.833208 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.833227 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0713 10:15:45.833257 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I0713 10:15:45.833346 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:45.833395 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0713 10:15:45.833523 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.833536 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-1 ok I0713 10:15:45.833579 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0713 10:15:45.833611 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0713 10:15:45.833620 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0713 10:15:45.833640 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0713 10:15:45.833631 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:45.833658 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0713 10:15:45.833679 13927 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0713 10:15:45.833686 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0713 10:15:45.833705 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0713 10:15:45.833722 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0713 10:15:45.833744 13927 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0713 10:15:45.833688 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:45.833750 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } W0713 10:15:45.833766 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:45.833782 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:45.833766 13927 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0713 10:15:45.833999 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedIP I0713 10:15:45.834148 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:45.834239 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:45.834262 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:45.834287 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:45.834313 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:45.834343 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:45.834368 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:45.834386 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:45.834409 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:45.834428 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:45.834458 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0713 10:15:45.853588 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:45.855590 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.855679 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.855721 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:45.925485 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:46.005499 13927 iamrole.go:89] ListRoles: { } I0713 10:15:46.007428 13927 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0713 10:15:46.007873 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:46.007893 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0713 10:15:46.008073 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:46.008098 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0713 10:15:46.008031 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.008176 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0713 10:15:46.008240 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0713 10:15:46.008258 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:46.008345 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:46.008375 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:46.008417 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0713 10:15:46.008423 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0713 10:15:46.008482 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0713 10:15:46.008580 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.008759 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:46.008931 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:46.009007 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:46.009103 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:46.009850 13927 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0713 10:15:46.009951 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0713 10:15:46.009918 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.009986 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:46.010017 13927 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0713 10:15:46.010010 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0713 10:15:46.010044 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.010099 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0713 10:15:46.010113 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0713 10:15:46.010079 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0713 10:15:46.010127 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0713 10:15:46.010153 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:46.010195 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0713 10:15:46.010167 13927 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-ip-eepmph", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-ip-eepmph" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0713 10:15:46.010250 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:46.010245 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0713 10:15:46.010293 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0713 10:15:46.010279 13927 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0713 10:15:46.010285 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0713 10:15:46.010364 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0713 10:15:46.010437 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0713 10:15:46.010468 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:46.010549 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:46.010580 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:46.010605 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0713 10:15:46.010718 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:46.010767 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:46.010796 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:46.010817 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0713 10:15:46.010922 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:46.010975 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:46.010999 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:46.011019 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.011150 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-12345678" } I0713 10:15:46.011167 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.011326 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0713 10:15:46.011318 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0713 10:15:46.011421 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:46.011453 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:46.011484 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0713 10:15:46.011530 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0713 10:15:46.011595 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:46.011627 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:46.011652 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:46.011691 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:46.011718 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:46.011737 13927 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0713 10:15:46.011522 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0713 10:15:46.011823 13927 executor.go:111] Tasks: 69 done / 116 total; 32 can run W0713 10:15:46.011910 13927 api.go:53] PageSize not implemented I0713 10:15:46.011951 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0713 10:15:46.012011 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0713 10:15:46.012033 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.012055 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.012114 13927 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } W0713 10:15:46.012085 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:46.012160 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.012180 13927 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0713 10:15:46.012213 13927 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0713 10:15:46.012140 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.012241 13927 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-ip-eepmph", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], Type: "network" } I0713 10:15:46.012267 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:46.012411 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.012534 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:46.012654 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0713 10:15:46.012780 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.012881 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:46.012973 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0713 10:15:46.013051 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.013158 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:46.013210 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:46.013280 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0713 10:15:46.013357 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:46.013408 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0713 10:15:46.013496 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:46.013539 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0713 10:15:46.013564 13927 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1", Port: 22, Protocol: "TCP" } W0713 10:15:46.013677 13927 api.go:53] PageSize not implemented I0713 10:15:46.013625 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.013697 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.013731 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.013815 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.013885 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.013960 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:46.014013 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0713 10:15:46.014054 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.014206 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0713 10:15:46.014361 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I0713 10:15:46.014492 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" }] }] } I0713 10:15:46.014636 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.014760 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.014882 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0713 10:15:46.015003 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0713 10:15:46.015055 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:46.015172 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.015267 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0713 10:15:46.015303 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0713 10:15:46.015362 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0713 10:15:46.015443 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.015518 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0713 10:15:46.015590 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:46.015619 13927 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0713 10:15:46.015688 13927 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0713 10:15:46.015715 13927 api.go:53] PageSize not implemented I0713 10:15:46.015743 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0713 10:15:46.015828 13927 api.go:53] PageSize not implemented I0713 10:15:46.015800 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:46.015871 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.015860 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:46.015889 13927 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:46.016098 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:46.016174 13927 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }] }] } I0713 10:15:46.016275 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0713 10:15:46.016316 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:46.016355 13927 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0713 10:15:46.018606 13927 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0713 10:15:46.018686 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:46.018830 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0713 10:15:46.018968 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.019088 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.019259 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.019389 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.021131 13927 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0713 10:15:46.021315 13927 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0713 10:15:46.021400 13927 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0713 10:15:46.021439 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:46.021475 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:46.021522 13927 dns.go:232] Pre-creating DNS records I0713 10:15:46.021546 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:46.025183 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:46.027157 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.027228 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.027278 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.098788 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:46.175344 13927 iamrole.go:89] ListRoles: { } I0713 10:15:46.176804 13927 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0713 10:15:46.176968 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.177341 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.177397 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:46.177516 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0713 10:15:46.177583 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0713 10:15:46.177603 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0713 10:15:46.177630 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:46.182129 13927 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0713 10:15:46.182553 13927 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0713 10:15:46.182573 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.182738 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0713 10:15:46.182688 13927 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0713 10:15:46.182786 13927 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0713 10:15:46.182798 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0713 10:15:46.182670 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0713 10:15:46.182843 13927 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0713 10:15:46.182918 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:46.182973 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.183165 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:46.183218 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.183349 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0713 10:15:46.183449 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0713 10:15:46.183494 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.183616 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0713 10:15:46.183780 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:46.183828 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0713 10:15:46.183937 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0713 10:15:46.184097 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:46.184153 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:46.184349 13927 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0713 10:15:46.184421 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:46.184485 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0713 10:15:46.184528 13927 api.go:53] PageSize not implemented I0713 10:15:46.184583 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0713 10:15:46.184577 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0713 10:15:46.184634 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:46.184650 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.184648 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0713 10:15:46.184675 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0713 10:15:46.184676 13927 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0713 10:15:46.184648 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:46.184695 13927 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0713 10:15:46.184718 13927 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0713 10:15:46.184739 13927 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } W0713 10:15:46.184787 13927 api.go:53] PageSize not implemented I0713 10:15:46.184797 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.184760 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.184852 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.185144 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:46.185226 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0713 10:15:46.185295 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:46.185560 13927 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0713 10:15:46.185589 13927 api.go:53] PageSize not implemented W0713 10:15:46.185602 13927 api.go:53] PageSize not implemented I0713 10:15:46.185633 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0713 10:15:46.185635 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:46.185694 13927 dnsname.go:118] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0713 10:15:46.185634 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.185707 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:46.185817 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.185941 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.188603 13927 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0713 10:15:46.188685 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.188784 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.188954 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.189091 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0713 10:15:46.189181 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.191823 13927 executor.go:111] Tasks: 112 done / 116 total; 2 can run W0713 10:15:46.191900 13927 api.go:53] PageSize not implemented I0713 10:15:46.191917 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.192082 13927 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0713 10:15:46.192157 13927 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0713 10:15:46.195283 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:46.195336 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:46.195367 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:46.195401 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:46.195433 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:46.195462 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:46.195498 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:46.195532 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:46.195567 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0713 10:15:46.195606 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0713 10:15:46.195653 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:46.195686 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:46.195718 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:46.195754 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:46.195784 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0713 10:15:46.195816 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:46.195847 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:46.195881 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0713 10:15:46.195915 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0713 10:15:46.195947 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:46.195983 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:46.196013 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0713 10:15:46.196046 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:46.196076 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:46.196104 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:46.196137 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:46.196169 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:46.197505 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:46.197563 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:46.197579 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:46.197617 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.197673 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0713 10:15:46.197731 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.197793 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.197830 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.197873 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.197913 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.197956 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0713 10:15:46.198004 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.198057 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:46.198083 13927 natgateway.go:117] DescribeNatGateways: { } I0713 10:15:46.198122 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0713 10:15:46.198157 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0713 10:15:46.198197 13927 api.go:53] PageSize not implemented I0713 10:15:46.198212 13927 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0713 10:15:46.198250 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:46.198273 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:46.198287 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0713 10:15:46.198322 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:46.198343 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:46.198360 13927 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0713 10:15:46.198416 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:46.198453 13927 iamrole.go:89] ListRoles: { } I0713 10:15:46.198498 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:46.198524 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:46.198567 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:46.198607 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:46.198636 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0713 10:15:46.198668 13927 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0713 10:15:46.198697 13927 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-config:lt-1 ok I0713 10:15:46.199272 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0713 10:15:46.199295 13927 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } autoscaling-group:nodes.private-shared-ip.example.com ok load-balancer:api-private-shared-ip-exa-ohatqj ok I0713 10:15:46.199320 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0713 10:15:46.199393 13927 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4 ok I0713 10:15:46.199406 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0713 10:15:46.199445 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0713 10:15:46.199442 13927 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0713 10:15:46.199478 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1 ok iam-instance-profile:nodes.private-shared-ip.example.com ok autoscaling-group:bastion.private-shared-ip.example.com ok I0713 10:15:46.199496 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0713 10:15:46.199337 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } I0713 10:15:46.199373 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } W0713 10:15:46.199547 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0713 10:15:46.199534 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0713 10:15:46.199562 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0713 10:15:46.199594 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:46.199642 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 } ] } W0713 10:15:46.199778 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:46.199796 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0713 10:15:46.199832 13927 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0713 10:15:46.199880 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:46.199927 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0713 10:15:46.199968 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0713 10:15:46.200070 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:46.200084 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0713 10:15:46.200111 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0713 10:15:46.200179 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:46.200249 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0713 10:15:46.200283 13927 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0713 10:15:46.200256 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0713 10:15:46.200293 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } subnet:subnet-1 ok I0713 10:15:46.200313 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0713 10:15:46.200317 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0713 10:15:46.200332 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0713 10:15:46.200356 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0713 10:15:46.200345 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:46.200363 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0713 10:15:46.200377 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0713 10:15:46.200394 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0713 10:15:46.200416 13927 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0713 10:15:46.200423 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0713 10:15:46.200439 13927 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0713 10:15:46.200376 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0713 10:15:46.200466 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:46.200474 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:46.200504 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:46.200533 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0713 10:15:46.200711 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.37s) === RUN TestLifecycleManyAddons I0713 10:15:46.200839 13927 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:46.200926 13927 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:46.200950 13927 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:46.200973 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:46.200995 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:46.201023 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:46.201046 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:46.201063 13927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:46.201086 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:46.201102 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:46.201129 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0713 10:15:46.219696 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:46.222391 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.222459 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:46.296180 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.044989 13927 iamrole.go:89] ListRoles: { } I0713 10:15:47.046085 13927 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0713 10:15:47.046325 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I0713 10:15:47.046378 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:47.046423 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0713 10:15:47.046525 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:47.046538 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0713 10:15:47.046699 13927 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.046940 13927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0713 10:15:47.046880 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:47.047033 13927 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" }] } I0713 10:15:47.047103 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:47.047166 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.047302 13927 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.047393 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:47.047431 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.047549 13927 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.047586 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.047739 13927 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0713 10:15:47.048600 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:47.048666 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0713 10:15:47.048704 13927 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0713 10:15:47.048747 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.048797 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.052395 13927 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0713 10:15:47.052521 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.052607 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0713 10:15:47.052710 13927 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0713 10:15:47.052700 13927 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0713 10:15:47.052753 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:47.052724 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0713 10:15:47.052832 13927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:47.052925 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.052966 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeTags\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:47.053039 13927 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:47.053074 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0713 10:15:47.053203 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I0713 10:15:47.053268 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.053408 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.053520 13927 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.053567 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:47.053597 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.053665 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.053695 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.053716 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0713 10:15:47.053786 13927 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0713 10:15:47.053817 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:47.053874 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:47.053909 13927 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0713 10:15:47.053945 13927 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0713 10:15:47.054025 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:47.054071 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:47.054133 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:47.054186 13927 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I0713 10:15:47.055487 13927 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"acm:DescribeCertificate\",\n \"acm:ListCertificates\",\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateTags\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeInternetGateways\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSnapshots\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcPeeringConnections\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListenerCertificates\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeRules\",\n \"elasticloadbalancing:DescribeTags\",\n \"elasticloadbalancing:DescribeTargetGroupAttributes\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteSnapshot\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddListenerCertificates\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteRule\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyRule\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:ModifyTargetGroupAttributes\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:RemoveListenerCertificates\",\n \"elasticloadbalancing:RemoveTags\",\n \"elasticloadbalancing:SetIpAddressType\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\",\n \"elasticloadbalancing:SetSecurityGroups\",\n \"elasticloadbalancing:SetSubnets\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateRule\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:47.056895 13927 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0713 10:15:47.057039 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.057139 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.057243 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.057346 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:47.057517 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.057621 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.057741 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.057827 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.057903 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:47.058043 13927 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:47.058093 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:47.058217 13927 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0713 10:15:47.058290 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:47.058353 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:47.058476 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.058560 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.058640 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.058698 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.058780 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0713 10:15:47.058998 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0713 10:15:47.059055 13927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0713 10:15:47.059097 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I0713 10:15:47.059236 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.059382 13927 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0713 10:15:47.059524 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0713 10:15:47.059680 13927 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I0713 10:15:47.059915 13927 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0713 10:15:47.060140 13927 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0713 10:15:47.060380 13927 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0713 10:15:47.060439 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.060478 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:47.060537 13927 dns.go:232] Pre-creating DNS records I0713 10:15:47.060566 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:47.066460 13927 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0713 10:15:47.069939 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.070032 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.139931 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.831538 13927 iamrole.go:89] ListRoles: { } I0713 10:15:47.832714 13927 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0713 10:15:47.832910 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.833112 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.833205 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.833234 13927 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.833282 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.833321 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I0713 10:15:47.833418 13927 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0713 10:15:47.833450 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.834259 13927 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0713 10:15:47.836199 13927 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0713 10:15:47.836336 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.836497 13927 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.836625 13927 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0713 10:15:47.836570 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.836707 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0713 10:15:47.836836 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.836938 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:47.837040 13927 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.838789 13927 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0713 10:15:47.839348 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.839545 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0713 10:15:47.839696 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:47.841810 13927 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0713 10:15:47.842030 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:47.842381 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:47.843346 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0713 10:15:47.843538 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.843702 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.843873 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.844025 13927 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0713 10:15:47.846253 13927 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0713 10:15:47.846558 13927 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0713 10:15:47.846785 13927 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0713 10:15:47.849733 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0713 10:15:47.849810 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0713 10:15:47.849846 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0713 10:15:47.849873 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0713 10:15:47.849905 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0713 10:15:47.849935 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0713 10:15:47.849963 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0713 10:15:47.849992 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0713 10:15:47.850022 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0713 10:15:47.850062 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0713 10:15:47.850097 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0713 10:15:47.850126 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0713 10:15:47.850153 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0713 10:15:47.850183 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0713 10:15:47.850212 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:47.850241 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0713 10:15:47.850272 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.850306 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0713 10:15:47.850334 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0713 10:15:47.850366 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0713 10:15:47.850394 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0713 10:15:47.850428 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0713 10:15:47.850461 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.850493 13927 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0713 10:15:47.852075 13927 delete_cluster.go:128] Looking for cloud resources to delete W0713 10:15:47.852141 13927 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0713 10:15:47.852157 13927 keypairs.go:95] DescribeKeyPairs: { } I0713 10:15:47.852194 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852247 13927 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0713 10:15:47.852296 13927 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852371 13927 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852423 13927 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852475 13927 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852517 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852556 13927 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0713 10:15:47.852599 13927 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852648 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0713 10:15:47.852679 13927 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0713 10:15:47.852711 13927 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0713 10:15:47.852742 13927 api.go:53] PageSize not implemented I0713 10:15:47.852757 13927 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0713 10:15:47.852778 13927 loadbalancers.go:35] PageSize not implemented I0713 10:15:47.852792 13927 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0713 10:15:47.852811 13927 targetgroups.go:35] PageSize not implemented I0713 10:15:47.852825 13927 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0713 10:15:47.852869 13927 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0713 10:15:47.852904 13927 iamrole.go:89] ListRoles: { } I0713 10:15:47.852943 13927 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.852966 13927 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0713 10:15:47.853004 13927 internetgateways.go:101] DescribeInternetGateways: { } I0713 10:15:47.853044 13927 routetable.go:53] DescribeRouteTables: { } I0713 10:15:47.853072 13927 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok I0713 10:15:47.863255 13927 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0713 10:15:47.863376 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0713 10:15:47.863440 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0713 10:15:47.863512 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0713 10:15:47.863747 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:47.863765 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0713 10:15:47.863809 13927 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0713 10:15:47.863855 13927 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0713 10:15:47.863961 13927 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0713 10:15:47.863973 13927 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0713 10:15:47.864018 13927 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok autoscaling-group:nodes.minimal.example.com ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0713 10:15:47.864242 13927 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0713 10:15:47.864301 13927 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0713 10:15:47.864355 13927 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0713 10:15:47.864428 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:47.864508 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0713 10:15:47.864563 13927 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:47.864608 13927 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0713 10:15:47.864724 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0713 10:15:47.864780 13927 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0713 10:15:47.864793 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0713 10:15:47.864826 13927 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0713 10:15:47.864856 13927 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0713 10:15:47.864898 13927 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0713 10:15:47.864909 13927 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0713 10:15:47.864934 13927 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0713 10:15:47.864685 13927 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0713 10:15:47.865066 13927 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0713 10:15:47.865164 13927 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0713 10:15:47.865258 13927 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0713 10:15:47.865567 13927 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleManyAddons (1.66s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 51.963s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.004s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0713 10:14:59.428270 14343 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.005s === RUN TestPodController I0713 10:14:59.459819 14346 pod.go:59] starting pod controller I0713 10:14:59.460185 14346 stoppable.go:62] shutting down controller I0713 10:14:59.460280 14346 pod.go:107] Got stop signal --- PASS: TestPodController (0.00s) PASS I0713 10:14:59.460327 14346 pod.go:65] shutting down pod controller ok k8s.io/kops/dns-controller/pkg/watchers 0.038s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.004s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00018bf40} route53_test.go:93: Got zone list: [0xc00018bf70] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc0001940b0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040e0a0} route53_test.go:93: Got zone list: [0xc00040e0d0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc0001940c8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00040e1d0} route53_test.go:170: Successfully added managed DNS zone: &{0xc00040a150 0xc0001940e8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040e390} route53_test.go:93: Got zone list: [0xc00040e3c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194108} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040e520} route53_test.go:93: Got zone list: [0xc00040e550] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194130} route53_test.go:187: Successfully added resource record set: {0xc0001e0600 0xc000194150} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040e830} route53_test.go:93: Got zone list: [0xc00040e860] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc0001941e0} route53_test.go:199: Successfully added resource record set: {0xc0001e0780 0xc000194200} route53_test.go:118: Got 1 recordsets: [0xc00040ea00] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040eb90} route53_test.go:93: Got zone list: [0xc00040ebc0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc0001942a0} route53_test.go:221: Successfully added resource record set: {0xc0001e0900 0xc0001942c0} route53_test.go:228: Correctly failed to add duplicate resource record {0xc0001e0900 0xc0001942c0}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040ef80} route53_test.go:93: Got zone list: [0xc00040efb0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194380} route53_test.go:246: Successfully removed resource set {0xc0001e0b00 0xc0001943a0} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040f290} route53_test.go:93: Got zone list: [0xc00040f2c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194430} route53_test.go:264: Successfully removed resource set {0xc0001e0c80 0xc000194450} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040f5d0} route53_test.go:93: Got zone list: [0xc00040f600] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc0001944e8} commontests.go:192: Successfully added resource record set: {0xc0001e0e00 0xc000194508} commontests.go:45: Correctly replaced resource record {0xc0001e0e00 0xc000194508} -> {0xc0001e0f00 0xc000194558} commontests.go:142: Got 1 recordsets: [0xc00040f910] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00040fb40} route53_test.go:93: Got zone list: [0xc00040fb70] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194610} commontests.go:192: Successfully added resource record set: {0xc0001e1180 0xc000194630} commontests.go:70: Correctly replaced resource record {0xc0001e1180 0xc000194630} -> {0xc0001e1280 0xc000194680} commontests.go:142: Got 1 recordsets: [0xc00040fe80] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00041e100} route53_test.go:93: Got zone list: [0xc00041e130] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194748} commontests.go:192: Successfully added resource record set: {0xc0001e1500 0xc000194768} commontests.go:142: Got 2 recordsets: [0xc00041e3a0 0xc00041e3c0] commontests.go:142: Got 2 recordsets: [0xc00041e430 0xc00041e450] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00041e650} route53_test.go:93: Got zone list: [0xc00041e680] route53_test.go:98: Got at least 1 zone in list:&{0xc0003cfef0 0xc000194870} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.013s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.005s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f8b00} clouddns_test.go:71: Got zone list: [0xc0004fc2e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fc288} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f8c00} clouddns_test.go:71: Got zone list: [0xc0004fc468] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fc408} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f8d00} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000358680 0xc0004fc5e8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f8dc0} clouddns_test.go:71: Got zone list: [0xc0004fc768] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fc708} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f95a0} clouddns_test.go:71: Got zone list: [0xc0004fc960] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fc900} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f96e0} clouddns_test.go:71: Got zone list: [0xc0004fcb40] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fcb10} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f9840} clouddns_test.go:71: Got zone list: [0xc0004fcdc8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fcd98} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f99a0} clouddns_test.go:71: Got zone list: [0xc0004fcf60] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fcf30} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f9ae0} clouddns_test.go:71: Got zone list: [0xc0004fd158] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fd128} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f9c40} clouddns_test.go:71: Got zone list: [0xc0004fd428] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fd2f0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f9da0} clouddns_test.go:71: Got zone list: [0xc0004fd758] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fd728} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc0002f9f20} clouddns_test.go:71: Got zone list: [0xc0004fda88] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fda58} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc0004ab770 0xc000114120} clouddns_test.go:71: Got zone list: [0xc0004fde60] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000358500 0xc0004fde30} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.058s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.020s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.073s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0713 10:15:07.015471 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.015638 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.015706 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.015758 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.015801 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.015868 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.015912 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.015938 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.015981 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.016013 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.029124 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0713 10:15:07.032166 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.032239 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.032261 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.032288 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.032327 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.032356 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.032377 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.032391 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.032412 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.032428 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.033438 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0713 10:15:07.039287 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.039391 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.039423 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.039461 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.039494 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.039534 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.039567 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.039593 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.039636 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.039659 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.041067 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.043017 14844 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0713 10:15:07.043047 14844 assetstore.go:111] bin/containerd I0713 10:15:07.043059 14844 assetstore.go:111] bin/containerd-shim I0713 10:15:07.043070 14844 assetstore.go:111] bin/containerd-shim-runc-v1 I0713 10:15:07.043079 14844 assetstore.go:111] bin/containerd-shim-runc-v2 I0713 10:15:07.043089 14844 assetstore.go:111] bin/containerd-stress I0713 10:15:07.043101 14844 assetstore.go:111] bin/ctr I0713 10:15:07.043158 14844 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0713 10:15:07.043176 14844 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0713 10:15:07.047396 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.047492 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.047525 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.047566 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.047595 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.047655 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.047687 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.047714 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.047746 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.047766 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.048784 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.050070 14844 containerd.go:60] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0713 10:15:07.050893 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.051032 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.051089 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.051144 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.051195 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.051253 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.051307 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.051355 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.051405 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.051444 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.052853 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.054348 14844 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0713 10:15:07.055035 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.055105 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.055127 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.055156 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.055176 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.055201 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.055245 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.055261 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.055281 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.055294 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.056304 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.057844 14844 assetstore.go:108] Matching assets for "^docker/": I0713 10:15:07.057864 14844 assetstore.go:111] docker/containerd I0713 10:15:07.057882 14844 assetstore.go:111] docker/containerd-shim I0713 10:15:07.057895 14844 assetstore.go:111] docker/ctr I0713 10:15:07.057909 14844 assetstore.go:111] docker/docker I0713 10:15:07.057917 14844 assetstore.go:111] docker/docker-init I0713 10:15:07.057923 14844 assetstore.go:111] docker/docker-proxy I0713 10:15:07.057929 14844 assetstore.go:111] docker/dockerd I0713 10:15:07.057935 14844 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0713 10:15:07.060741 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.060826 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.060850 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.060882 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.060901 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.060924 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.060944 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.060959 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.060977 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.061001 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.062189 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.064442 14844 assetstore.go:108] Matching assets for "^docker/": I0713 10:15:07.064462 14844 assetstore.go:111] docker/containerd I0713 10:15:07.064475 14844 assetstore.go:111] docker/containerd-shim I0713 10:15:07.064486 14844 assetstore.go:111] docker/ctr I0713 10:15:07.064497 14844 assetstore.go:111] docker/docker I0713 10:15:07.064508 14844 assetstore.go:111] docker/docker-init I0713 10:15:07.064519 14844 assetstore.go:111] docker/docker-proxy I0713 10:15:07.064529 14844 assetstore.go:111] docker/dockerd I0713 10:15:07.064540 14844 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0713 10:15:07.072572 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.072700 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.072736 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.072789 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.072827 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.072881 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.072919 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.072945 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.072983 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.073006 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.074580 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.076822 14844 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0713 10:15:07.077037 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.077138 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.077168 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.077205 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.077237 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.077279 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.077313 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.077340 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.077372 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.077394 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.078945 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:07.081026 14844 assetstore.go:108] Matching assets for "^docker/": I0713 10:15:07.081047 14844 assetstore.go:111] docker/docker I0713 10:15:07.081061 14844 assetstore.go:111] docker/docker-containerd I0713 10:15:07.081069 14844 assetstore.go:111] docker/docker-containerd-ctr I0713 10:15:07.081075 14844 assetstore.go:111] docker/docker-containerd-shim I0713 10:15:07.081081 14844 assetstore.go:111] docker/docker-init I0713 10:15:07.081088 14844 assetstore.go:111] docker/docker-proxy I0713 10:15:07.081094 14844 assetstore.go:111] docker/docker-runc I0713 10:15:07.081099 14844 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0713 10:15:07.086573 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.086673 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.086703 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.086741 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.086768 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.086798 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.086826 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.086845 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.086874 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.086892 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.089811 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0713 10:15:07.092925 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.093029 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.093065 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.093112 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.093149 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.093197 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.093233 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.093263 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.093303 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.093330 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.095433 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I0713 10:15:07.098203 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.098302 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.098337 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.098384 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.098422 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.098466 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.098501 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.098534 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.098573 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.098598 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.101659 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0713 10:15:07.109912 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.110034 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.110071 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.110116 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.110151 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.110201 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.110236 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.110265 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.110307 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.110332 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.112518 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I0713 10:15:07.127774 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.127902 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.127947 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.127998 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.128040 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.128085 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.128123 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.128154 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.128194 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.128223 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.131321 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.02s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0713 10:15:07.143752 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.143912 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.143982 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.144058 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.144130 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.144209 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.144283 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.144355 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.144432 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.144491 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.147586 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0713 10:15:07.157272 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.157428 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.157468 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.157514 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.157554 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.157601 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.157638 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.157667 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.157707 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.157733 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.159885 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0713 10:15:07.171918 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:07.172015 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:07.172046 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:07.172080 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:07.172105 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:07.172137 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:07.172164 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:07.172182 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:07.172204 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:07.172221 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:07.174291 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (0.98s) === RUN TestKubeAPIServerBuilderARM64 I0713 10:15:08.147079 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.147221 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.147264 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.147316 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.147354 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.147406 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.147449 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.147477 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.147521 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.147549 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.150504 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.13s) === RUN TestKubeControllerManagerBuilder I0713 10:15:08.281802 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.281936 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.281975 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.282019 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.282057 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.282106 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.282144 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.282173 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.282208 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.282232 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.285027 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.02s) === RUN TestKubeControllerManagerBuilderAMD64 I0713 10:15:08.298324 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.298429 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.298461 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.298496 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.298521 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.298556 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.298584 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.298603 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.298627 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.298644 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.300578 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.13s) === RUN TestKubeControllerManagerBuilderARM64 I0713 10:15:08.428643 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.428771 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.428809 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.428863 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.428900 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.428946 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.429001 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.429051 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.429088 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.429112 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.432036 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.14s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0713 10:15:08.565699 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.565848 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.565891 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.565939 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.565985 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.566038 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.566080 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.566115 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.566156 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.566184 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.569293 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0713 10:15:08.578638 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.578738 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.578770 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.578805 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.578830 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.578862 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.578890 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.578909 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.578932 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.578950 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.581023 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.13s) === RUN TestKubeProxyBuilderARM64 I0713 10:15:08.707421 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.707517 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.707547 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.707580 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.707605 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.707634 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.707659 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.707677 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.707701 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.707718 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.709484 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.13s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0713 10:15:08.842754 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.842886 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.842923 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.842970 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.843008 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.843054 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.843092 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.843120 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.843159 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.843188 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.846109 14844 zones.go:93] ListHostedZonesPages { } W0713 10:15:08.850050 14844 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0713 10:15:08.852660 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.852778 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.852817 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.852857 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.852891 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.852936 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.852978 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.853007 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.853042 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.853069 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.855867 14844 zones.go:93] ListHostedZonesPages { } W0713 10:15:08.979640 14844 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.13s) === RUN TestKubeSchedulerBuilderARM64 I0713 10:15:08.984408 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:08.984570 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:08.984616 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:08.984668 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:08.984710 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:08.984767 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:08.984815 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:08.984850 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:08.984893 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:08.984922 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:08.987494 14844 zones.go:93] ListHostedZonesPages { } W0713 10:15:09.110040 14844 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.13s) === RUN TestKubectlBuilder I0713 10:15:09.113077 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.113214 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.113259 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.113311 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.113358 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.113412 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.113458 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.113492 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.113536 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.113566 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.116636 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:09.119560 14844 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0713 10:15:09.122667 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.122774 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.122837 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.122886 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.122930 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.122982 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.123023 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.123058 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.123100 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.123130 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.125692 14844 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0713 10:15:09.131338 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.131398 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.131433 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.131465 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.131493 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.131521 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.131541 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.131558 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.131580 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.131595 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.132830 14844 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0713 10:15:09.136110 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.136203 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.136225 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.136248 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.136270 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.136296 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.136317 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.136336 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.136358 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.136373 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.154107 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:09.157103 14844 assetstore.go:108] Matching assets for "protokube$": I0713 10:15:09.157129 14844 assetstore.go:111] /opt/kops/bin/protokube I0713 10:15:09.157145 14844 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0713 10:15:09.157182 14844 assetstore.go:108] Matching assets for "channels$": I0713 10:15:09.157200 14844 assetstore.go:111] /opt/kops/bin/channels I0713 10:15:09.157213 14844 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.02s) === RUN TestSecretBuilder I0713 10:15:09.159484 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.159607 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.159643 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.159685 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.159728 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.159778 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.159818 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.159849 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.159888 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.159915 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.162842 14844 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.01s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0713 10:15:09.166514 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.166612 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.166642 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.166690 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.166727 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.167379 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.167445 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.167496 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.167540 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.167574 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.765928 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:09.769289 14844 update_service.go:91] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.60s) === RUN TestUpdateServiceBuilderExternal I0713 10:15:09.770172 14844 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:09.770314 14844 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:09.770353 14844 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:09.770446 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:09.770522 14844 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:09.770577 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:09.770623 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:09.770692 14844 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:09.770739 14844 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:09.770768 14844 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:09.774004 14844 zones.go:93] ListHostedZonesPages { } I0713 10:15:09.777110 14844 update_service.go:86] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 2.841s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.028s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.032s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.030s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.024s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0713 10:15:06.963085 14852 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0713 10:15:06.963315 14852 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.014s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0713 10:15:07.016503 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.016615 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.016664 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.016706 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.016737 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0713 10:15:07.016986 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017029 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017055 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017080 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017116 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017142 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017177 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017200 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017235 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017277 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017305 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017337 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017383 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017449 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.017479 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.020834 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.020877 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.020946 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0713 10:15:07.020995 14853 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0713 10:15:07.021277 14853 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0713 10:15:07.021363 14853 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.081s === RUN TestManagedFieldsMigrator === RUN TestManagedFieldsMigrator/csa-migration === RUN TestManagedFieldsMigrator/merge-owners === RUN TestManagedFieldsMigrator/ssa-unchanged === RUN TestManagedFieldsMigrator/unknown-unchanged --- PASS: TestManagedFieldsMigrator (0.01s) --- PASS: TestManagedFieldsMigrator/csa-migration (0.00s) --- PASS: TestManagedFieldsMigrator/merge-owners (0.00s) --- PASS: TestManagedFieldsMigrator/ssa-unchanged (0.00s) --- PASS: TestManagedFieldsMigrator/unknown-unchanged (0.00s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.026s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0713 10:15:07.083181 14863 builder.go:230] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.10s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0713 10:15:07.084239 14863 builder.go:230] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0713 10:15:07.085067 14863 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0713 10:15:07.085822 14863 builder.go:230] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0713 10:15:07.086462 14863 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0713 10:15:07.087107 14863 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0713 10:15:07.087959 14863 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0713 10:15:07.088482 14863 copyfile.go:208] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0713 10:15:07.088494 14863 copyfile.go:209] S3, Google Cloud Storage, and File Paths are supported. E0713 10:15:07.088499 14863 copyfile.go:210] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0713 10:15:07.088505 14863 copyfile.go:211] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.139s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.003s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.058s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.020s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.086s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.023s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.005s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.004s === RUN TestHasExtraFields === RUN TestHasExtraFields/minimal === RUN TestHasExtraFields/extraFields === RUN TestHasExtraFields/spec2 === RUN TestHasExtraFields/isolateMasters === RUN TestHasExtraFields/instanceGroup --- PASS: TestHasExtraFields (0.01s) --- PASS: TestHasExtraFields/minimal (0.01s) --- PASS: TestHasExtraFields/extraFields (0.00s) --- PASS: TestHasExtraFields/spec2 (0.00s) --- PASS: TestHasExtraFields/isolateMasters (0.00s) --- PASS: TestHasExtraFields/instanceGroup (0.00s) PASS ok k8s.io/kops/pkg/edit 0.034s === RUN TestFlagToFalse I0713 10:15:14.698498 16281 featureflag_test.go:32] Created flag Unittest1 I0713 10:15:14.698548 16281 featureflag.go:166] FeatureFlag "UnitTest1"=false I0713 10:15:14.698560 16281 featureflag.go:166] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0713 10:15:14.698630 16281 featureflag.go:166] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) === RUN TestGetPositive --- PASS: TestGetPositive (0.00s) === RUN TestGetNegative --- PASS: TestGetNegative (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.005s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.028s === RUN TestRenderInstanceGroupZones W0713 10:15:14.719748 16283 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.027s === RUN TestWarmPoolOnlyRoll I0713 10:15:27.849821 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.849963 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.849997 16767 instancegroups.go:136] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0713 10:15:27.850813 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:27.850855 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.852042 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.852091 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.857334 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.857402 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.857444 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.857468 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:27.863639 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:27.864762 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:27.864914 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:27.864969 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.866142 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.866183 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.871347 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.871422 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:27.872648 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:27.872740 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:27.872785 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.873951 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.873979 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.879222 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.879278 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.879308 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.879330 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:27.880843 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:27.882011 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:27.882069 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.886105 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.886148 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.891418 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.891527 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:27.892911 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:27.892965 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.894114 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.894153 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.899402 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.899488 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:27.900395 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:27.900433 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.901573 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.901602 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.906872 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.906917 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.906938 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.906956 16767 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0713 10:15:27.907914 16767 instancegroups.go:431] Draining the node: "node-2a.local". I0713 10:15:27.908683 16767 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0713 10:15:27.908722 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.910008 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.910042 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.915317 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.915418 16767 instancegroups.go:431] Draining the node: "node-2b.local". I0713 10:15:27.916870 16767 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0713 10:15:27.916927 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.918199 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.918229 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.923491 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.923601 16767 instancegroups.go:431] Draining the node: "node-2c.local". I0713 10:15:27.925036 16767 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0713 10:15:27.925177 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.926415 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.926458 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.931705 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.931797 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.08s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0713 10:15:27.932632 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:27.932679 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.933891 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.933945 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.933979 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.933991 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:27.934015 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.935162 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.935216 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.935240 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:27.935307 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.936500 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.936545 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.936597 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.936628 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:27.936666 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.937839 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.937884 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.937897 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:27.937920 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.939036 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.939115 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.939141 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:27.939181 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.940363 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.940383 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.940411 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.940419 16767 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0713 10:15:27.940441 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.941589 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.941648 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.941670 16767 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0713 10:15:27.941706 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.942834 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0713 10:15:27.942882 16767 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0713 10:15:27.942901 16767 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0713 10:15:27.942925 16767 instancegroups.go:467] waiting for 1ms after terminating instance W0713 10:15:27.944096 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0713 10:15:27.944137 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0713 10:15:27.944650 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:27.944689 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.945824 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.945858 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.945875 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.945921 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.945944 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.945957 16767 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.945969 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:27.946755 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:27.947398 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:27.947468 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:27.947491 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.948637 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.948681 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.948709 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.948762 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:27.949888 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:27.949967 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:27.950009 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.951178 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.951209 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.951227 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.951246 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.951262 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.951273 16767 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.951285 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:27.952140 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:27.953098 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:27.953140 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.954287 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.954326 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.954342 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.954371 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:27.955068 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:27.955097 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.956240 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.956290 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.956318 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.956369 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:27.957531 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:27.957576 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.958704 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.958737 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.958754 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.958772 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.958798 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.958814 16767 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.958827 16767 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0713 10:15:27.959849 16767 instancegroups.go:431] Draining the node: "node-2a.local". I0713 10:15:27.960506 16767 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0713 10:15:27.960536 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.961670 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.961704 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.961721 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.961751 16767 instancegroups.go:431] Draining the node: "node-2b.local". I0713 10:15:27.962944 16767 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0713 10:15:27.962974 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.964091 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.964114 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.964127 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.964154 16767 instancegroups.go:431] Draining the node: "node-2c.local". I0713 10:15:27.964664 16767 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0713 10:15:27.964684 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.965794 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.965823 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0713 10:15:27.965835 16767 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0713 10:15:27.965850 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0713 10:15:27.966071 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0713 10:15:27.966236 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:27.966254 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.967369 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.967386 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.972581 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.972667 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.972688 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.972703 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:27.973287 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:27.973731 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:27.973790 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:27.973813 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.974925 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.974949 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.980183 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.980257 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:27.981102 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:27.981177 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:27.981211 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.982335 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.982363 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.987568 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.987619 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.987641 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.987664 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:27.988587 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:27.989165 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:27.989195 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.990322 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.990358 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:27.995591 16767 instancegroups.go:537] Cluster validated. I0713 10:15:27.995681 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:27.996647 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:27.996682 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:27.997807 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:27.997833 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.003053 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.003122 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.004080 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.004120 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.005271 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.005294 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.010514 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.010561 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.010588 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.010608 16767 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0713 10:15:28.011755 16767 instancegroups.go:431] Draining the node: "node-2a.local". I0713 10:15:28.012544 16767 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0713 10:15:28.012585 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.013722 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.013761 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.019016 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.019093 16767 instancegroups.go:431] Draining the node: "node-2b.local". I0713 10:15:28.020044 16767 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0713 10:15:28.020083 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.021215 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.021245 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.026453 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.026519 16767 instancegroups.go:431] Draining the node: "node-2c.local". I0713 10:15:28.027427 16767 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0713 10:15:28.027466 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.028593 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.028619 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.033815 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.033857 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0713 10:15:28.034034 16767 rollingupdate.go:111] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0713 10:15:28.034368 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:28.034390 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.035530 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.035565 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.035585 16767 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0713 10:15:28.035914 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:28.035940 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.037081 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.037110 16767 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0713 10:15:28.037124 16767 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0713 10:15:28.037388 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.037406 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.037416 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.037425 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0713 10:15:28.037536 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.037550 16767 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0713 10:15:28.037560 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.037568 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0713 10:15:28.037762 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:28.037779 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.038891 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.038923 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.044145 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.044205 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.044221 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.044230 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:28.044933 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:28.045689 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:28.045779 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:28.045813 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.046941 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.046977 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.046999 16767 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0713 10:15:28.047369 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:28.047398 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.048556 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.048579 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.053813 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.053872 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.053890 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.053899 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:28.054418 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:28.054842 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:28.054899 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:28.054920 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.056041 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.056075 16767 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0713 10:15:28.056096 16767 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0713 10:15:28.056427 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.056455 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.056470 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.057545 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.058306 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.058336 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.059459 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.059486 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.064709 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.064753 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.065359 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.065396 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.067620 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.067654 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.072852 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.072899 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.073506 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.073532 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.074652 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.074680 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.079925 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.079964 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0713 10:15:28.080813 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.080834 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.080847 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.080860 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0713 10:15:28.081015 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.081033 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.081042 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.081051 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0713 10:15:28.081176 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.081192 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.081200 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.081209 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupExitableError I0713 10:15:28.081360 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.081375 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.081383 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.082028 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.082490 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.082513 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.083631 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.083656 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.089029 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.089080 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.089694 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.089720 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.090838 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.090865 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.096060 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.096110 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.096693 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.096718 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.097837 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.097864 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.103067 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.103097 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.103114 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.103132 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.103144 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-2": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupExitableError (0.02s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0713 10:15:28.103350 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.103367 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.103375 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.104014 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.104468 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.104491 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.105612 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.105657 16767 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0713 10:15:28.105679 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.105699 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0713 10:15:28.105870 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.105902 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.105914 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.106854 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.107568 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.107600 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.108747 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.108782 16767 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0713 10:15:28.108816 16767 instancegroups.go:506] Cluster did not validate within 0s E0713 10:15:28.108833 16767 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0713 10:15:28.109132 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:28.109167 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.110318 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.110378 16767 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0713 10:15:28.111496 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.116776 16767 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0713 10:15:28.117895 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.123128 16767 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0713 10:15:28.124307 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.129555 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.129656 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.129689 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.129710 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:28.130612 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:28.131440 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:28.131526 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:28.131564 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.132704 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.132745 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.137969 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.138038 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:28.139032 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:28.139120 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:28.139159 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.140295 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.140358 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.145608 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.145681 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.145729 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.145751 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.146947 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.147838 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.147878 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.149047 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.149104 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.154353 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.154437 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.155540 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.155582 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.156760 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.156808 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.162037 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.162112 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.163136 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.163173 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.164311 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.164355 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.169681 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.169726 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.169757 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.169774 16767 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0713 10:15:28.170789 16767 instancegroups.go:431] Draining the node: "node-2a.local". I0713 10:15:28.171516 16767 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0713 10:15:28.171550 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.172677 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.172718 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.177946 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.178018 16767 instancegroups.go:431] Draining the node: "node-2b.local". I0713 10:15:28.178999 16767 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0713 10:15:28.179037 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.180169 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.180211 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.185458 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.185541 16767 instancegroups.go:431] Draining the node: "node-2c.local". I0713 10:15:28.186522 16767 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0713 10:15:28.186564 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.187703 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.187752 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.192999 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.193040 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0713 10:15:28.193320 16767 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0713 10:15:28.193342 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.194465 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.194503 16767 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0713 10:15:28.195631 16767 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0713 10:15:28.196776 16767 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0713 10:15:28.197895 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.203121 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.203208 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.203237 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.203257 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:28.204072 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:28.204806 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:28.204882 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:28.204917 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.206047 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.206078 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.211283 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.211348 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:28.212231 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:28.212317 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:28.212355 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.213487 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.213523 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.218724 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.218771 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.218793 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.218812 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.219880 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.220676 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.220713 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.221840 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.221877 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.227080 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.227169 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.228151 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.228187 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.229337 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.229362 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.235726 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.235801 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.236670 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.236707 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.237867 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.237901 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.243115 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.243172 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.243190 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.243201 16767 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0713 10:15:28.244488 16767 instancegroups.go:431] Draining the node: "node-2a.local". I0713 10:15:28.244932 16767 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0713 10:15:28.244959 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.246078 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.246101 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.251289 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.251343 16767 instancegroups.go:431] Draining the node: "node-2b.local". I0713 10:15:28.251911 16767 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0713 10:15:28.251938 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.253081 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.253103 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.258282 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.258336 16767 instancegroups.go:431] Draining the node: "node-2c.local". I0713 10:15:28.258917 16767 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0713 10:15:28.258946 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.260086 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.260108 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.265300 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.265344 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0713 10:15:28.266788 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.266838 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.266865 16767 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0713 10:15:28.268186 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.268761 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.268786 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.269951 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.269980 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.275214 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.275259 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.275802 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.275825 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.276947 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.276975 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.282241 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.282268 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0713 10:15:28.282513 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.282530 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.282538 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:28.282977 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:28.283385 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:28.283426 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:28.283442 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.284562 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.284586 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.289776 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.289806 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:28.290256 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:28.290294 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:28.290311 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.291427 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.291451 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.296675 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.296723 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.01s) === RUN TestRollingUpdateDisabled I0713 10:15:28.297083 16767 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled W0713 10:15:28.297123 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0713 10:15:28.297146 16767 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled W0713 10:15:28.297165 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0713 10:15:28.297190 16767 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled W0713 10:15:28.297204 16767 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0713 10:15:28.297223 16767 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0713 10:15:28.297241 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0713 10:15:28.297450 16767 instancegroups.go:604] Detaching instance "bastion-1a", in group "bastion-1". I0713 10:15:28.297501 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:28.298611 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.298648 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.303897 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.303947 16767 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled I0713 10:15:28.304018 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.304043 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.304058 16767 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0713 10:15:28.304807 16767 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled I0713 10:15:28.304830 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.304848 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.304863 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:28.305694 16767 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0713 10:15:28.305754 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:28.306889 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.306912 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.312131 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.312181 16767 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled I0713 10:15:28.312203 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.312231 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.312249 16767 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0713 10:15:28.313318 16767 instancegroups.go:602] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0713 10:15:28.313384 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:28.314536 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.314556 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:28.319771 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.319813 16767 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0713 10:15:28.319838 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0713 10:15:28.320179 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.320213 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.320229 16767 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0713 10:15:28.322398 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.323192 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.323239 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.324414 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.324458 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.324491 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.324635 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.325414 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.326046 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.326085 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.327236 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.327276 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.430451 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.430518 16767 instancegroups.go:431] Draining the node: "node-1e.local". I0713 10:15:28.430570 16767 instancegroups.go:431] Draining the node: "node-1d.local". I0713 10:15:28.431529 16767 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0713 10:15:28.431802 16767 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0713 10:15:28.431853 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.432991 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.433028 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.433065 16767 instancegroups.go:431] Draining the node: "node-1f.local". I0713 10:15:28.433582 16767 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0713 10:15:28.433610 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.434747 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.434809 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.535178 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.535244 16767 instancegroups.go:431] Draining the node: "node-1g.local". I0713 10:15:28.535882 16767 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0713 10:15:28.535921 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.537049 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.537087 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.537107 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0713 10:15:28.537361 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.537396 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.537414 16767 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0713 10:15:28.538791 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.538860 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.539297 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.539633 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.539661 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.540792 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.540839 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.641203 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.641261 16767 instancegroups.go:431] Draining the node: "node-1d.local". I0713 10:15:28.641331 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:28.641950 16767 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0713 10:15:28.642414 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:28.642453 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.643600 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.643633 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.643664 16767 instancegroups.go:431] Draining the node: "node-1e.local". I0713 10:15:28.644124 16767 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0713 10:15:28.644150 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.645267 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.645319 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.745681 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.745752 16767 instancegroups.go:431] Draining the node: "node-1f.local". I0713 10:15:28.746425 16767 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0713 10:15:28.746463 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.747582 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.747625 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.747647 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0713 10:15:28.747880 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.747906 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.747920 16767 instancegroups.go:342] Tainting 7 nodes in "master-1" instancegroup. I0713 10:15:28.749888 16767 instancegroups.go:431] Draining the node: "master-1a.local". I0713 10:15:28.750552 16767 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0713 10:15:28.750615 16767 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0713 10:15:28.750655 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.751818 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.751849 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.751879 16767 instancegroups.go:431] Draining the node: "master-1c.local". I0713 10:15:28.751895 16767 instancegroups.go:431] Draining the node: "master-1b.local". I0713 10:15:28.752689 16767 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0713 10:15:28.752735 16767 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0713 10:15:28.752877 16767 instancegroups.go:688] skipping deregistration of instance "master-1c", as part of instancegroup with role "ControlPlane" I0713 10:15:28.752908 16767 instancegroups.go:625] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0713 10:15:28.752929 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.754041 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.754085 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.854486 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.854571 16767 instancegroups.go:431] Draining the node: "master-1e.local". I0713 10:15:28.854585 16767 instancegroups.go:431] Draining the node: "master-1d.local". I0713 10:15:28.855759 16767 instancegroups.go:688] skipping deregistration of instance "master-1e", as part of instancegroup with role "ControlPlane" I0713 10:15:28.855840 16767 instancegroups.go:625] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0713 10:15:28.856023 16767 instancegroups.go:688] skipping deregistration of instance "master-1d", as part of instancegroup with role "ControlPlane" I0713 10:15:28.856071 16767 instancegroups.go:625] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0713 10:15:28.856111 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.857236 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.857273 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.857307 16767 instancegroups.go:431] Draining the node: "master-1f.local". I0713 10:15:28.858060 16767 instancegroups.go:688] skipping deregistration of instance "master-1f", as part of instancegroup with role "ControlPlane" I0713 10:15:28.858110 16767 instancegroups.go:625] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0713 10:15:28.858145 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.871661 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.871761 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.972191 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.972255 16767 instancegroups.go:431] Draining the node: "master-1g.local". I0713 10:15:28.972859 16767 instancegroups.go:688] skipping deregistration of instance "master-1g", as part of instancegroup with role "ControlPlane" I0713 10:15:28.972917 16767 instancegroups.go:625] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0713 10:15:28.972951 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.974098 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.974129 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.974143 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.23s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0713 10:15:28.974375 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.974397 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.974406 16767 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0713 10:15:28.975412 16767 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0713 10:15:28.975457 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:28.976599 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.976628 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.976636 16767 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0713 10:15:28.976668 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:28.977799 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.977824 16767 instancegroups.go:537] Cluster validated. I0713 10:15:28.977864 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:28.977873 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:28.978741 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:28.978922 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:28.978951 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:28.980101 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:28.980137 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.080562 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.080626 16767 instancegroups.go:431] Draining the node: "node-1d.local". I0713 10:15:29.080650 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:29.081655 16767 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0713 10:15:29.081915 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:29.081962 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.083121 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.083159 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.083193 16767 instancegroups.go:431] Draining the node: "node-1e.local". I0713 10:15:29.083889 16767 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0713 10:15:29.083931 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.085056 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.085120 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.185572 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.185656 16767 instancegroups.go:431] Draining the node: "node-1f.local". I0713 10:15:29.186570 16767 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0713 10:15:29.186666 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.187831 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.187871 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.187891 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0713 10:15:29.188097 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.188117 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.188125 16767 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0713 10:15:29.189188 16767 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0713 10:15:29.189223 16767 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0713 10:15:29.189249 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:29.190377 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.190402 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.190437 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:29.190450 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:29.191288 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:29.191450 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:29.191477 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.192615 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.192653 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.293105 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.293170 16767 instancegroups.go:431] Draining the node: "node-1d.local". I0713 10:15:29.293215 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:29.294061 16767 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0713 10:15:29.294227 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:29.294256 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.295385 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.295422 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.295457 16767 instancegroups.go:431] Draining the node: "node-1e.local". I0713 10:15:29.296239 16767 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0713 10:15:29.296287 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.297421 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.297487 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.397741 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.397813 16767 instancegroups.go:431] Draining the node: "node-1f.local". I0713 10:15:29.398671 16767 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0713 10:15:29.398724 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.399851 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.399886 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.399906 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0713 10:15:29.400137 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.400159 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.400170 16767 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0713 10:15:29.400746 16767 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0713 10:15:29.400772 16767 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0713 10:15:29.400804 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:29.401944 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.401964 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.407149 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.407204 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:29.407223 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:29.408272 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:29.408323 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.408481 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:29.408511 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.411087 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.411134 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.416355 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.416399 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0713 10:15:29.416722 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.416748 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.416761 16767 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0713 10:15:29.417507 16767 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0713 10:15:29.417564 16767 instancegroups.go:191] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0713 10:15:29.417579 16767 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0713 10:15:29.417617 16767 instancegroups.go:191] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0713 10:15:29.417647 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:29.418770 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.418798 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.424007 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.424079 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:29.424091 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:29.425177 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:29.425213 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.425407 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:29.425440 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.426350 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.426377 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.431589 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.431619 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0713 10:15:29.431865 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.431896 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.431911 16767 instancegroups.go:342] Tainting 4 nodes in "node-1" instancegroup. I0713 10:15:29.433071 16767 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0713 10:15:29.433129 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:29.434246 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.434282 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.434293 16767 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0713 10:15:29.434359 16767 instancegroups.go:203] waiting for 1ms after detaching instance I0713 10:15:29.436410 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.436441 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.436482 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:29.436497 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:29.436541 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:29.437247 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:29.437290 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.438038 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:29.438074 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.438286 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:29.438324 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.438350 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.538737 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.538820 16767 instancegroups.go:431] Draining the node: "node-1d.local". I0713 10:15:29.539614 16767 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0713 10:15:29.539653 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.540873 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.540912 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.540935 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0713 10:15:29.541182 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.541214 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.541230 16767 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0713 10:15:29.543295 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:29.543958 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:29.543992 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.545124 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.545160 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.545193 16767 instancegroups.go:431] Draining the node: "node-1e.local". I0713 10:15:29.545219 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:29.546102 16767 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0713 10:15:29.546327 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:29.546363 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.547512 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.547572 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.648294 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.648355 16767 instancegroups.go:431] Draining the node: "node-1g.local". I0713 10:15:29.648414 16767 instancegroups.go:431] Draining the node: "node-1f.local". I0713 10:15:29.648998 16767 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0713 10:15:29.649461 16767 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0713 10:15:29.649506 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.650664 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.650691 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.650715 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:29.651212 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:29.651238 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.652339 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.652384 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.752801 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.752875 16767 instancegroups.go:431] Draining the node: "node-1d.local". I0713 10:15:29.753486 16767 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0713 10:15:29.753529 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.754965 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.754993 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.755007 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0713 10:15:29.755172 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.755186 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.755195 16767 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0713 10:15:29.755206 16767 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0713 10:15:29.755217 16767 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0713 10:15:29.755230 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0713 10:15:29.755350 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.755365 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.755373 16767 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0713 10:15:29.757314 16767 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0713 10:15:29.757355 16767 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0713 10:15:29.757364 16767 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0713 10:15:29.757402 16767 instancegroups.go:431] Draining the node: "node-1a.local". I0713 10:15:29.757901 16767 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0713 10:15:29.757923 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.759016 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.759044 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.764176 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.764218 16767 instancegroups.go:431] Draining the node: "node-1b.local". I0713 10:15:29.764888 16767 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0713 10:15:29.764918 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.766061 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.766082 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.771285 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.771343 16767 instancegroups.go:431] Draining the node: "node-1c.local". I0713 10:15:29.771948 16767 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0713 10:15:29.771973 16767 instancegroups.go:467] waiting for 1ms after terminating instance I0713 10:15:29.773124 16767 instancegroups.go:501] Validating the cluster. I0713 10:15:29.773145 16767 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0713 10:15:29.778375 16767 instancegroups.go:537] Cluster validated. I0713 10:15:29.778422 16767 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 2.033s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.004s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.012s === RUN TestParse E0713 10:15:27.764435 16770 versions.go:96] unable to parse Kubernetes version "" E0713 10:15:27.764559 16770 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.010s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.029s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option --- PASS: TestBuildKubecfg (0.17s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option (0.00s) PASS ok k8s.io/kops/pkg/kubeconfig 0.252s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.03s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.115s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.095s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.073s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0713 10:15:27.820037 16781 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.071s === RUN Test_RunEtcdManagerBuilder I0713 10:15:27.832142 16782 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0713 10:15:27.875377 16782 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 W0713 10:15:27.877348 16782 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.131s === RUN Test_RunKubeApiserverBuilder I0713 10:15:27.833150 16784 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.103s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.106s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.026s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.064s === RUN TestServerGroupBuilder I0713 10:15:36.042994 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.059168 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.150282 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.165735 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.176897 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.198720 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.264653 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.280283 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.295425 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.305116 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.317585 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.324517 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.338312 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.346437 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.354401 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.362110 17668 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.40s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.484s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.026s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.010s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.025s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.77s) --- PASS: TestIssueCert/ca (0.88s) --- PASS: TestIssueCert/client (0.42s) --- PASS: TestIssueCert/clientOneYear (0.47s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.790s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.005s === RUN TestAddUntaggedRouteTables I0713 10:15:36.049417 17674 routetable.go:53] DescribeRouteTables: { } W0713 10:15:36.049607 17674 tags.go:67] cluster tag not found on route-table:rtb-1234 I0713 10:15:36.049639 17674 aws.go:229] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0713 10:15:36.049835 17674 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0713 10:15:36.049964 17674 aws.go:2059] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0713 10:15:36.050094 17674 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0713 10:15:36.050243 17674 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0713 10:15:36.050368 17674 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0713 10:15:36.050510 17674 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0713 10:15:36.050644 17674 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0713 10:15:36.050693 17674 volumes.go:128] DescribeVolumes: { } W0713 10:15:36.050737 17674 aws.go:2212] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0713 10:15:36.050954 17674 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:36.051070 17674 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:36.051114 17674 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:36.051222 17674 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:36.051271 17674 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:36.051354 17674 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:36.051442 17674 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:36.051482 17674 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:36.051549 17674 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:36.051606 17674 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:36.051641 17674 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0713 10:15:36.051682 17674 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0713 10:15:36.051722 17674 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0713 10:15:36.051815 17674 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0713 10:15:36.051856 17674 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0713 10:15:36.051923 17674 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0713 10:15:36.051994 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0713 10:15:36.052039 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0713 10:15:36.052081 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0713 10:15:36.052119 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0713 10:15:36.052163 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0713 10:15:36.052202 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0713 10:15:36.052249 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0713 10:15:36.052284 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0713 10:15:36.052353 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0713 10:15:36.052397 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0713 10:15:36.052456 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0713 10:15:36.052503 17674 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.084s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.030s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.073s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.007s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.003s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.003s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.003s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.003s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.048s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.096s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.005s === RUN TestRemovesDuplicateGuardedBlocks W0713 10:15:44.201606 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201660 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201673 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201685 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201697 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201711 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201724 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201734 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201741 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201749 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201759 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201770 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201784 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201796 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.201807 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.201819 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218247 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218273 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218338 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218355 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218427 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218447 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218531 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218547 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218627 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218644 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218727 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218744 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218839 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218856 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.218962 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.218988 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219074 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219093 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219168 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219180 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219251 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219263 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219315 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219326 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219390 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219402 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219472 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219483 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219538 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219548 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219603 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219614 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219666 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219682 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219734 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219748 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219802 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219814 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.219935 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.219951 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220009 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220023 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220093 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220108 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220161 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220174 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220231 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220248 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220337 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220366 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220451 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220463 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220552 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220570 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220645 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220659 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220741 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220757 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220834 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220852 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.220955 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.220973 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221058 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221074 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221149 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221166 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221244 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221261 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221337 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221354 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221445 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221481 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221569 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221586 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221666 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221692 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221781 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221801 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.221904 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.221926 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222045 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222065 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222157 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222176 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222276 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222295 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222429 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222450 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222518 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222529 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222583 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222592 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222646 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222656 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222712 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222725 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222777 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222788 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222839 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222850 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222902 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222913 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.222963 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.222974 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223028 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223039 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223119 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223138 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223224 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223242 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223325 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223345 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223432 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223450 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223537 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223555 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223635 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223652 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223736 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223754 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223843 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223863 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.223959 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.223978 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224099 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224119 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224212 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224231 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224322 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224340 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224429 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224446 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224554 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224573 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224660 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224678 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224769 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224788 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.224908 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.224932 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225016 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225034 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225127 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225145 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225238 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225258 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225351 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225370 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225461 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225479 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225563 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225582 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225682 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225701 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225793 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225813 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.225916 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.225936 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.226029 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.226050 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.226147 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.226166 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.226261 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.226282 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.226372 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.226392 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227174 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227193 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227277 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227291 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227360 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227375 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227441 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227457 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227546 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227565 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227657 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227677 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227796 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227815 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.227908 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.227928 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228015 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228035 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228124 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228141 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228225 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228242 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228333 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228352 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228443 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228461 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228545 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228561 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228642 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228660 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.228748 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.228767 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.03s) === RUN TestRecoversFromBadNesting W0713 10:15:44.229267 19053 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0713 10:15:44.229283 19053 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0713 10:15:44.229295 19053 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0713 10:15:44.229306 19053 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0713 10:15:44.229319 19053 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0713 10:15:44.229330 19053 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0713 10:15:44.229339 19053 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0713 10:15:44.229375 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229393 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229406 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.229423 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229436 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.229452 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229464 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.229477 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229490 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.229503 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229516 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229530 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229542 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229555 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.229571 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.229584 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243011 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243049 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243150 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243173 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243266 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243285 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243378 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243395 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243526 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243543 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243631 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243651 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243771 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243789 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243879 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243896 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.243982 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.243997 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244084 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244100 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244192 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244209 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244306 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244324 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244417 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244434 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244520 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244538 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244626 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244643 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244732 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244751 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244842 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244858 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.244957 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.244974 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245067 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245083 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245175 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245192 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245287 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245303 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245391 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245409 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245501 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245518 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245606 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245621 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245720 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245738 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245832 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245849 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.245941 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.245958 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246051 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246067 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246195 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246213 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246302 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246319 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246406 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246424 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246510 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246531 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246619 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246637 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246728 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246746 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246836 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246852 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.246942 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.246959 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247048 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247065 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247151 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247171 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247260 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247277 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247364 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247381 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247468 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247486 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247573 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247590 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247706 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247724 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247817 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247833 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.247921 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.247937 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248020 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248036 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248121 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248138 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248219 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248235 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248322 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248339 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248428 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248445 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248533 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248550 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248641 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248658 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248798 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248816 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.248908 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.248925 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249028 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249043 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249131 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249150 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249237 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249259 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249347 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249365 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249456 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249476 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249591 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249611 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249699 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249718 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249806 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249824 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.249916 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.249936 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250041 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250061 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250155 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250174 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250264 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250283 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250370 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250389 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250477 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250496 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250583 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250602 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250689 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250715 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250804 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250825 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.250910 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.250927 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251009 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251027 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251111 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251128 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251205 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251225 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251322 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251341 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251471 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251490 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251576 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251593 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251674 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251695 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251781 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251800 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251890 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.251908 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.251998 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.252060 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.252184 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.252235 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.252381 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.252436 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.252561 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.252614 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.252742 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.252793 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.252913 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.252964 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253099 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253121 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253225 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253247 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253340 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253359 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253451 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253470 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253565 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253583 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253671 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253689 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253785 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253803 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.253893 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.253912 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.254003 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.254021 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.254107 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.254124 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.254211 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.254229 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0713 10:15:44.254312 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0713 10:15:44.254330 19053 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.03s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.057s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.031s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.062s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.042s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0713 10:15:44.229242 19059 files.go:57] Writing file "/tmp/TestWriteFile3766831133/001/SubDir/test1.tmp" I0713 10:15:44.229395 19059 files.go:113] Changing file mode for "/tmp/TestWriteFile3766831133/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.039s === RUN TestBootstrapChannelBuilder_BuildTasks I0713 10:15:47.401816 19781 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:47.402005 19781 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:47.402056 19781 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:47.402115 19781 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:47.402161 19781 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:47.402229 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:47.402289 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:47.402330 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:47.402379 19781 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:47.402420 19781 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:47.414304 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.465439 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.586374 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.675999 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.758583 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.835307 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:47.930764 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:48.015614 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:48.138394 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:48.694633 19781 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.35s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0713 10:15:48.750946 19781 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:48.751030 19781 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:48.751075 19781 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:48.751108 19781 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:48.751135 19781 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:48.751169 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:48.751195 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:48.751211 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:48.751234 19781 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:48.751252 19781 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:48.751271 19781 featureflag.go:169] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0713 10:15:48.752426 19781 zones.go:93] ListHostedZonesPages { } I0713 10:15:48.850512 19781 featureflag.go:169] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.10s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0713 10:15:48.850645 19781 featureflag.go:166] FeatureFlag "ImageDigest"=false I0713 10:15:48.850719 19781 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:48.850766 19781 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0713 10:15:48.850799 19781 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0713 10:15:48.850827 19781 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0713 10:15:48.850858 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0713 10:15:48.850885 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0713 10:15:48.850902 19781 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0713 10:15:48.850924 19781 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0713 10:15:48.850941 19781 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0713 10:15:48.852174 19781 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.09s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.12s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0713 10:15:49.108302 19781 new_cluster.go:1132] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestDefaultImage --- PASS: TestDefaultImage (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.01s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0713 10:15:49.149008 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149046 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149060 19781 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0713 10:15:49.149080 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0713 10:15:49.149092 19781 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0713 10:15:49.149110 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0713 10:15:49.149142 19781 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0713 10:15:49.149166 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149179 19781 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0713 10:15:49.149196 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149208 19781 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0713 10:15:49.149233 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149262 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149282 19781 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0713 10:15:49.149296 19781 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0713 10:15:49.149315 19781 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0713 10:15:49.149330 19781 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0713 10:15:49.149342 19781 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0713 10:15:49.149353 19781 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0713 10:15:49.149365 19781 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0713 10:15:49.149375 19781 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0713 10:15:49.150704 19781 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN TestKopsFeatureEnabled === RUN TestKopsFeatureEnabled/Missing_feature === RUN TestKopsFeatureEnabled/Existing_feature I0713 10:15:49.150924 19781 featureflag.go:166] FeatureFlag "Scaleway"=true --- PASS: TestKopsFeatureEnabled (0.00s) --- PASS: TestKopsFeatureEnabled/Missing_feature (0.00s) --- PASS: TestKopsFeatureEnabled/Existing_feature (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.841s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0713 10:15:47.380824 19782 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender359831758/001 I0713 10:15:47.381104 19782 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender359831758/001 I0713 10:15:47.381347 19782 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender359831758/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0713 10:15:47.385662 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0713 10:15:47.385752 19782 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0713 10:15:47.385807 19782 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0713 10:15:47.385912 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.386053 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.386113 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.386130 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.386280 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.386322 19782 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0713 10:15:47.386355 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.386418 19782 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0713 10:15:47.386495 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0713 10:15:47.386547 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.386586 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.386633 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.386657 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.386704 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.386756 19782 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0713 10:15:47.386836 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0713 10:15:47.387026 19782 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0713 10:15:47.387086 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.387232 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0713 10:15:47.387286 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.387317 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.387358 19782 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0713 10:15:47.387434 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.387604 19782 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0713 10:15:47.387647 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.387672 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.387691 19782 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0713 10:15:47.387737 19782 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.388046 19782 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0713 10:15:47.388198 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.388271 19782 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0713 10:15:47.388354 19782 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0713 10:15:47.388465 19782 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0713 10:15:47.388552 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.388648 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.388707 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.388774 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.388837 19782 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0713 10:15:47.388918 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.389040 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.389103 19782 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0713 10:15:47.389183 19782 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.389295 19782 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0713 10:15:47.389359 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0713 10:15:47.389430 19782 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0713 10:15:47.389594 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0713 10:15:47.389683 19782 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0713 10:15:47.389748 19782 internetgateways.go:66] CreateInternetGateway: { } I0713 10:15:47.389803 19782 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0713 10:15:47.389914 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.390003 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.390044 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.390058 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.390130 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.390157 19782 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0713 10:15:47.390187 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.390258 19782 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0713 10:15:47.390336 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0713 10:15:47.390384 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.390419 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.390454 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.390468 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.390516 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.390552 19782 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0713 10:15:47.390602 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0713 10:15:47.391102 19782 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender318989227/001 I0713 10:15:47.391351 19782 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender318989227/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0713 10:15:47.391759 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.391810 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.391895 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0713 10:15:47.391931 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.391962 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.391988 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.392061 19782 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0713 10:15:47.392253 19782 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0713 10:15:47.392300 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0713 10:15:47.392351 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0713 10:15:47.392413 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.392472 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.392514 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.392531 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.392553 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.392577 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.392627 19782 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0713 10:15:47.392707 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0713 10:15:47.392960 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.393006 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.393083 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0713 10:15:47.393130 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.393150 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.393176 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.393233 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.393325 19782 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0713 10:15:47.393363 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.393382 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.393405 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0713 10:15:47.393463 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.393507 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.393547 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.393561 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.393584 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.393608 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.393652 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.393704 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.393729 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0713 10:15:47.393835 19782 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0713 10:15:47.393889 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.393998 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0713 10:15:47.394070 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.394094 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.394119 19782 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0713 10:15:47.394184 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.394332 19782 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0713 10:15:47.394399 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.394573 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.395234 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.395260 19782 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:47.395294 19782 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0713 10:15:47.395366 19782 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0713 10:15:47.395412 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.395455 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.395469 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.395492 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.395520 19782 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0713 10:15:47.395575 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.395661 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.395733 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.395752 19782 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0713 10:15:47.395861 19782 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0713 10:15:47.395909 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.395982 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0713 10:15:47.396013 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.396032 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.396054 19782 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0713 10:15:47.396098 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.396183 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.396316 19782 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0713 10:15:47.396372 19782 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0713 10:15:47.396438 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.396479 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.396513 19782 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0713 10:15:47.396632 19782 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0713 10:15:47.396683 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.396736 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.396755 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.396783 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.396809 19782 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0713 10:15:47.396850 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.396898 19782 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0713 10:15:47.397014 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.397042 19782 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0713 10:15:47.397121 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0713 10:15:47.397186 19782 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0713 10:15:47.397256 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.397294 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.397329 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.397344 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.397417 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.397443 19782 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0713 10:15:47.397474 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.397510 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.397610 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0713 10:15:47.397634 19782 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0713 10:15:47.397663 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0713 10:15:47.397728 19782 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0713 10:15:47.397765 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.397800 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.397815 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.397846 19782 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0713 10:15:47.397882 19782 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0713 10:15:47.397934 19782 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0713 10:15:47.398015 19782 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:47.398052 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.398126 19782 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0713 10:15:47.398159 19782 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0713 10:15:47.398180 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.398201 19782 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:47.398237 19782 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:47.398272 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0713 10:15:47.398310 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.398325 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.398352 19782 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0713 10:15:47.398374 19782 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0713 10:15:47.398479 19782 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0713 10:15:47.398539 19782 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:47.398576 19782 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0713 10:15:47.398613 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0713 10:15:47.398628 19782 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0713 10:15:47.398656 19782 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.067s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.047s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.025s === RUN TestDiskRenderAzure I0713 10:15:47.628465 19910 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0713 10:15:47.629237 19910 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0713 10:15:47.629711 19910 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:124: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0713 10:15:47.630494 19910 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0713 10:15:47.630948 19910 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:102: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0713 10:15:47.631413 19910 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0713 10:15:47.631885 19910 resourcegroup.go:108] Creating a new Resource Group with name: rg I0713 10:15:47.631909 19910 resourcegroup.go:110] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0713 10:15:47.632268 19910 resourcegroup.go:108] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0713 10:15:47.633131 19910 subnet.go:107] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0713 10:15:47.633515 19910 virtualnetwork.go:122] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0713 10:15:47.633889 19910 virtualnetwork.go:122] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0713 10:15:47.634471 19910 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0713 10:15:47.634920 19910 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.040s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.035s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scaleway [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scalewaytasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/install [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestProjectIAMBinding I0713 10:15:50.637831 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.638686 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:50.638719 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.639314 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:50.639348 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.639539 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0713 10:15:50.639683 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.639778 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:50.639798 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.639915 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:50.639938 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.639983 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0713 10:15:50.640123 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.640421 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:50.640446 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.640750 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0713 10:15:50.640775 20810 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0713 10:15:50.640893 20810 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.522s ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN Test_OpenstackCloud_MakeCloud === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value --- PASS: Test_OpenstackCloud_MakeCloud (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value (0.00s) === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) === RUN Test_BuildClients === RUN Test_BuildClients/Empty_openstack_spec_means_no_load_balancer === RUN Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned === RUN Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned === RUN Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned --- PASS: Test_BuildClients (0.00s) --- PASS: Test_BuildClients/Empty_openstack_spec_means_no_load_balancer (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned (0.00s) === RUN Test_WaitForStatusActiveIsSuccessful --- PASS: Test_WaitForStatusActiveIsSuccessful (1.00s) === RUN Test_WaitForStatusActiveResultsInInstanceNotFound --- PASS: Test_WaitForStatusActiveResultsInInstanceNotFound (1.00s) === RUN Test_WaitForStatusActiveResultsInUnableToCreateServer --- PASS: Test_WaitForStatusActiveResultsInUnableToCreateServer (1.00s) === RUN Test_WaitForStatusActiveResultsInTimeout --- PASS: Test_WaitForStatusActiveResultsInTimeout (1.00s) === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 4.537s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.519s === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.516s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.514s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.516s === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0713 10:15:52.255184 21079 bindmount.go:223] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0713 10:15:52.255243 21079 bindmount.go:232] running mount command [mount -o remount,exec /containerized_mounter] I0713 10:15:52.255255 21079 bindmount.go:223] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0713 10:15:52.255267 21079 bindmount.go:243] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0713 10:15:52.255277 21079 bindmount.go:223] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0713 10:15:52.255301 21079 bindmount.go:223] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0713 10:15:52.255319 21079 bindmount.go:223] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0713 10:15:52.255817 21079 service.go:110] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.063s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.005s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.005s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.004s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.005s === RUN TestGetProxyEnvVars W0713 10:15:52.218422 21086 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.022s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.008s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.005s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.003s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.020s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0713 10:15:56.667610 21179 s3context.go:235] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.054s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 137.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.26.3-r1.apk >>> kops: Build complete at Thu, 13 Jul 2023 10:16:01 +0000 elapsed time 0h 3m 39s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/5) Purging .makedepends-kops (20230713.101224) (2/5) Purging kubernetes (1.27.3-r1) (3/5) Purging go (1.20.6-r0) (4/5) Purging bash (5.2.15-r5) Executing bash-5.2.15-r5.pre-deinstall (5/5) Purging readline (8.2.1-r2) Executing busybox-1.36.1-r3.trigger OK: 376 MiB in 105 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...