>>> kops: Building testing/kops 1.26.6-r0 (using abuild 3.11.21-r0) started Sun, 24 Sep 2023 12:47:28 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go bash (1/5) Installing kubernetes (1.28.2-r0) (2/5) Installing go (1.21.1-r0) (3/5) Installing readline (8.2.1-r2) (4/5) Installing bash (5.2.15-r5) Executing bash-5.2.15-r5.post-install (5/5) Installing .makedepends-kops (20230924.124730) Executing busybox-1.36.1-r8.trigger OK: 556 MiB in 115 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Cleaning up tmpdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.6.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.26.6.tar.gz::https://github.com/kubernetes/kops/archive/v1.26.6.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 13007 0 13007 0 0 27258 0 --:--:-- --:--:-- --:--:-- 27258 100 11.0M 0 11.0M 0 0 7908k 0 --:--:-- 0:00:01 --:--:-- 11.5M 100 16.6M 0 16.6M 0 0 7034k 0 --:--:-- 0:00:02 --:--:-- 8747k 100 22.7M 0 22.7M 0 0 6779k 0 --:--:-- 0:00:03 --:--:-- 7865k 100 27.9M 0 27.9M 0 0 6462k 0 --:--:-- 0:00:04 --:--:-- 7237k 100 31.3M 0 31.3M 0 0 6329k 0 --:--:-- 0:00:05 --:--:-- 6985k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.26.6.tar.gz >>> kops: Checking sha512sums... kops-1.26.6.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.26.6.tar.gz... fatal: not a git repository (or any of the parent directories): .git mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.26.6/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.26.6/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version=1.26.6 -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops go: downloading github.com/google/go-containerregistry v0.13.0 go: downloading golang.org/x/crypto v0.7.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading k8s.io/apimachinery v0.26.2 go: downloading go.uber.org/multierr v1.10.0 go: downloading k8s.io/api v0.26.2 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.1 go: downloading github.com/spf13/viper v1.15.0 go: downloading helm.sh/helm/v3 v3.11.2 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading k8s.io/cli-runtime v0.26.2 go: downloading k8s.io/client-go v0.26.2 go: downloading k8s.io/klog/v2 v2.90.1 go: downloading github.com/aws/aws-sdk-go v1.44.217 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/kubectl v0.26.2 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/cert-manager/cert-manager v1.11.0 go: downloading golang.org/x/net v0.8.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/Masterminds/sprig/v3 v3.2.3 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading golang.org/x/sync v0.1.0 go: downloading k8s.io/cloud-provider-aws v1.26.1 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/gophercloud/gophercloud v1.2.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading golang.org/x/oauth2 v0.6.0 go: downloading google.golang.org/api v0.111.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.3 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/imdario/mergo v0.3.13 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/utils v0.0.0-20230308161112-d77c459e9343 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/sergi/go-diff v1.3.1 go: downloading sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd go: downloading k8s.io/kube-openapi v0.0.0-20221207184640-f3cff1453715 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading github.com/Azure/azure-sdk-for-go v68.0.0+incompatible go: downloading github.com/digitalocean/godo v1.97.0 go: downloading github.com/hetznercloud/hcloud-go v1.41.0 go: downloading github.com/scaleway/scaleway-sdk-go v1.0.0-beta.14 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.12 go: downloading k8s.io/mount-utils v0.26.2 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.2.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/huandu/xstrings v1.4.0 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.3.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/docker/cli v20.10.21+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading golang.org/x/sys v0.6.0 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading golang.org/x/term v0.6.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.334.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading golang.org/x/exp v0.0.0-20230307190834-24139beb5833 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/cloud-provider v0.26.2 go: downloading k8s.io/component-base v0.26.2 go: downloading k8s.io/csi-translation-lib v0.26.0 go: downloading github.com/kr/fs v0.1.0 go: downloading golang.org/x/time v0.3.0 go: downloading github.com/golang/protobuf v1.5.2 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/subosito/gotenv v1.4.2 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/pelletier/go-toml/v2 v2.0.6 go: downloading github.com/magiconair/properties v1.8.7 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading k8s.io/apiserver v0.26.2 go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading golang.org/x/text v0.8.0 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading cloud.google.com/go/compute/metadata v0.2.3 go: downloading k8s.io/cloud-provider-gcp/providers v0.26.2 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading cloud.google.com/go/compute v1.18.0 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/moby/sys/mountinfo v0.6.2 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/moby/term v0.0.0-20221205130635-1aeaba878587 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading k8s.io/apiextensions-apiserver v0.26.1 go: downloading sigs.k8s.io/gateway-api v0.6.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/mattn/go-isatty v0.0.17 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/containerd/containerd v1.6.15 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.2 go: downloading github.com/docker/docker v20.10.21+incompatible go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.2.0 go: downloading github.com/prometheus/common v0.39.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/go-openapi/jsonreference v0.20.0 go: downloading github.com/go-openapi/swag v0.22.3 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4 go: downloading github.com/emicklei/go-restful/v3 v3.10.1 go: downloading github.com/go-errors/errors v1.0.1 go: downloading google.golang.org/grpc v1.53.0 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/mailru/easyjson v0.7.7 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/gorilla/mux v1.8.0 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading google.golang.org/genproto v0.0.0-20230223222841-637eb2293923 go: downloading go.opencensus.io v0.24.0 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git go test -v ./... go: downloading sigs.k8s.io/controller-runtime v0.14.5 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/stretchr/testify v1.8.2 go: downloading k8s.io/kubelet v0.26.2 go: downloading github.com/google/go-tpm-tools v0.3.10 go: downloading github.com/google/go-tpm v0.3.3 go: downloading k8s.io/gengo v0.0.0-20230306165830-ab3349d207d4 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading golang.org/x/tools v0.6.0 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/google/go-sev-guest v0.4.1 go: downloading github.com/pborman/uuid v1.2.0 go: downloading github.com/google/logger v1.1.1 go: downloading golang.org/x/mod v0.8.0 ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0924 12:50:04.720912 62845 addon.go:233] addon test wants to update all nodes I0924 12:50:04.723450 62845 addon.go:233] addon test wants to update workers nodes I0924 12:50:04.723799 62845 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0924 12:50:04.724345 62845 addon.go:266] installing PKI for "test" I0924 12:50:05.142320 62845 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (0.88s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 0.953s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.100s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kopscontrollerclient [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/model/scalewaymodel [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/nodeidentity/scaleway [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/scaleway [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/gossip/scaleway [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scaleway [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/scalewaytasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/install [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestCreateClusterMinimal I0924 12:50:12.029049 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.029281 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.029331 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.029398 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.029456 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.029539 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.029597 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.029630 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.029675 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.029735 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.035617 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.035701 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.099452 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.099724 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.100263 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.120978 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.121064 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.121092 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.121125 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.121148 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.121183 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.121210 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.121227 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.121249 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.121274 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.125555 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.125623 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.163332 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.163533 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.164181 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.173151 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.173258 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.173293 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.173339 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.173374 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.173424 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.173459 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.173490 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.173529 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.173554 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.177331 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.177387 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.237964 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.238141 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.238429 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.244401 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.244485 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.244512 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.244542 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.244566 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.244616 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.244644 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.244662 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.244688 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.244707 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.302764 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.302817 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.366019 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.366211 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.366523 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.374752 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.374850 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.374882 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.374917 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.374943 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.374976 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.375007 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.375025 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.375051 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.375069 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.378905 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.378956 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.451097 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.451404 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.451849 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.459279 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.459384 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.459434 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.459480 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.459518 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.459573 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.459612 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.459645 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.459684 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.459718 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.463664 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.463724 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.501118 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.501307 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.501600 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.506989 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.507063 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.507087 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.507120 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.507145 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.507176 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.507205 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.507224 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.507247 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.507265 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.511059 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.511097 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.549993 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.550166 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.550481 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.555949 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.556038 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.556067 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.556100 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.556124 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.556156 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.556183 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.556202 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.556225 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.556244 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.560221 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.560277 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.651468 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.651586 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.651880 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:12.657214 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.657295 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.657327 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.657370 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.657404 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.657450 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.657485 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.657514 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.657554 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.657579 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.705053 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.705104 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.787022 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:12.787193 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:12.787494 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.76s) === RUN TestCreateClusterHetzner I0924 12:50:12.793407 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.793519 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.793544 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.793573 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.793595 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.793627 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.793652 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.793669 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.793693 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.793710 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.797291 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.797336 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0924 12:50:12.839280 65034 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0924 12:50:12.839330 65034 new_cluster.go:1354] Cloud Provider ID: "hetzner" I0924 12:50:12.846287 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.846367 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.846392 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.846421 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.846449 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.846484 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.846514 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.846532 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.846554 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.846571 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.849790 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.849829 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.903299 65034 new_cluster.go:1354] Cloud Provider ID: "hetzner" --- PASS: TestCreateClusterHetzner (0.12s) === RUN TestCreateClusterOpenStack I0924 12:50:12.910969 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.911050 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.911076 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.911107 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.911132 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.911163 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.911188 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.911205 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.911228 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.911244 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.916587 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.916655 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0924 12:50:12.941709 65034 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0924 12:50:12.941756 65034 new_cluster.go:1354] Cloud Provider ID: "openstack" I0924 12:50:12.945318 65034 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 --- PASS: TestCreateClusterOpenStack (0.05s) === RUN TestCreateClusterOpenStackOctavia I0924 12:50:12.957704 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:12.957856 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:12.957899 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:12.957986 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:12.958014 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:12.958078 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:12.958136 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:12.958165 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:12.958219 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:12.958273 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:12.962936 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:12.963010 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0924 12:50:12.998071 65034 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0924 12:50:12.998117 65034 new_cluster.go:1354] Cloud Provider ID: "openstack" I0924 12:50:13.000302 65034 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0924 12:50:13.000325 65034 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackOctavia (0.05s) === RUN TestCreateClusterOpenStackNoDNS I0924 12:50:13.008690 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.008789 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.008820 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.008852 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.008876 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.008910 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.008938 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.008958 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.008982 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.008999 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.012363 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.012402 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0924 12:50:13.059900 65034 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0924 12:50:13.059944 65034 new_cluster.go:1354] Cloud Provider ID: "openstack" I0924 12:50:13.060003 65034 subnets.go:185] Assigned CIDR 10.0.32.0/19 to subnet us-test1 I0924 12:50:13.060020 65034 subnets.go:199] Assigned CIDR 10.0.0.0/22 to subnet utility-us-test1 --- PASS: TestCreateClusterOpenStackNoDNS (0.06s) === RUN TestCreateClusterCilium I0924 12:50:13.069914 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.070014 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.070044 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.070076 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.070103 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.070146 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.070177 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.070203 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.070229 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.070247 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.074052 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.074116 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.123143 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.123312 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.123704 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterCilium (0.06s) === RUN TestCreateClusterOverride I0924 12:50:13.129442 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.129525 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.129551 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.129581 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.129605 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.129637 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.129666 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.129736 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.129761 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.129777 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.133577 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.133627 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.214163 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.214656 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.214938 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.09s) === RUN TestCreateClusterKubernetesFeatureGates I0924 12:50:13.220428 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.220513 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.220539 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.220568 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.220594 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.220625 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.220653 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.220671 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.220695 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.220712 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.226281 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.226377 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.258312 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.258477 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.258838 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterKubernetesFeatureGates (0.04s) === RUN TestCreateClusterComplex I0924 12:50:13.265199 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.265290 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.265316 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.265345 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.265379 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.265411 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.265437 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.265456 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.265479 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.265497 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.308708 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.308778 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.338028 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.338214 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.338548 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.08s) === RUN TestCreateClusterHA I0924 12:50:13.344275 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.344365 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.344393 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.344425 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.344452 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.344484 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.344514 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.344537 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.344564 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.344583 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.348992 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.349039 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.386906 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.387479 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.387509 65034 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0924 12:50:13.387527 65034 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0924 12:50:13.388091 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:13.401551 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.401676 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.401715 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.401758 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.401801 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.401851 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.401891 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.401919 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.401957 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.401987 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.406202 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.406269 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.469674 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.469986 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.470003 65034 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0924 12:50:13.470014 65034 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0924 12:50:13.470331 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.13s) === RUN TestCreateClusterMinimalGCE I0924 12:50:13.478558 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.478647 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.478674 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.478708 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.478733 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.478765 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.478810 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.478827 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.478851 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.478868 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.482900 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.482948 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.553459 65034 new_cluster.go:1354] Cloud Provider ID: "gce" I0924 12:50:13.553574 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 I0924 12:50:13.559017 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.559088 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.559112 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.559139 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.559165 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.559197 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.559223 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.559242 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.559265 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.559283 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.564930 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.565019 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.626029 65034 new_cluster.go:1354] Cloud Provider ID: "gce" I0924 12:50:13.626129 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterMinimalGCE (0.15s) === RUN TestCreateClusterHAGCE I0924 12:50:13.632606 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.632691 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.632718 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.632745 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.632772 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.632806 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.632837 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.632857 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.632882 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.632901 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.637483 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.637540 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.697469 65034 new_cluster.go:1354] Cloud Provider ID: "gce" I0924 12:50:13.697585 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.07s) === RUN TestCreateClusterGCE I0924 12:50:13.706085 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.706163 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.706189 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.706220 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.706244 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.706274 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.706300 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.706317 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.706342 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.706359 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.711767 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.711827 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.768827 65034 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0924 12:50:13.768887 65034 new_cluster.go:1354] Cloud Provider ID: "gce" I0924 12:50:13.768947 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.07s) === RUN TestCreateClusterHASharedZone I0924 12:50:13.774787 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.774854 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.774875 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.774898 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.774920 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.774948 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.774971 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.774988 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.775013 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.775028 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.778505 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.778540 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0924 12:50:13.887260 65034 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0924 12:50:13.887294 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:13.887512 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:13.887760 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.13s) === RUN TestCreateClusterHASharedZones I0924 12:50:13.900566 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:13.900715 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:13.900757 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:13.900807 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:13.900849 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:13.900900 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:13.900948 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:13.900980 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:13.901019 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:13.901048 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:13.906252 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:13.906297 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0924 12:50:14.027087 65034 new_cluster.go:924] Running with control-plane nodes in the same AZs; redundancy will be reduced I0924 12:50:14.027134 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.027483 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:14.027499 65034 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0924 12:50:14.027830 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.14s) === RUN TestCreateClusterPrivate I0924 12:50:14.036890 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.036972 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.036996 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.037027 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.037050 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.037081 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.037108 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.037128 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.037153 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.037171 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.040800 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.040848 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.074134 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.074327 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:14.074344 65034 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0924 12:50:14.074603 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:14.074642 65034 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.04s) === RUN TestCreateClusterPrivateGCE I0924 12:50:14.080670 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.080738 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.080757 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.080782 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.080807 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.080834 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.080858 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.080879 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.080903 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.080918 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.084209 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.084249 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.117935 65034 new_cluster.go:565] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0924 12:50:14.118011 65034 new_cluster.go:1354] Cloud Provider ID: "gce" I0924 12:50:14.118103 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.04s) === RUN TestCreateClusterWithNGWSpecified I0924 12:50:14.124465 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.124545 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.124571 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.124609 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.124634 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.124665 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.124693 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.124713 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.124735 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.124752 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.128341 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.128382 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.182344 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.182537 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:14.182555 65034 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0924 12:50:14.182841 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:14.182872 65034 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.06s) === RUN TestCreateClusterWithINGWSpecified I0924 12:50:14.188806 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.188886 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.188909 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.188937 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.188963 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.188994 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.189021 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.189039 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.189063 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.189080 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.192788 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.192840 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.241044 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.241248 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:14.241277 65034 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0924 12:50:14.241563 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:14.241594 65034 utils.go:213] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.06s) === RUN TestCreateClusterSharedVPC I0924 12:50:14.247530 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.247612 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.247636 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.247666 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.247691 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.247722 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.247748 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.247766 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.247788 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.247805 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.251114 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.251151 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.298826 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.298973 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.299012 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.299050 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.299066 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.299105 65034 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0924 12:50:14.299345 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.06s) === RUN TestCreateClusterSharedSubnets I0924 12:50:14.307169 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.307249 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.307273 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.307302 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.307327 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.307357 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.307385 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.307403 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.307424 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.307441 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.311305 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.311357 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.348067 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.348110 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.348195 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.348333 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.348356 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.348381 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.348401 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.348687 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.05s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0924 12:50:14.353750 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.353828 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.353853 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.353879 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.353903 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.353934 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.353960 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.353979 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.354003 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.354020 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.357491 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.357531 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.461440 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:14.461489 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.461514 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.461594 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.461728 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.461750 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.461775 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.461792 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.462088 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.11s) === RUN TestCreateClusterPrivateSharedSubnets I0924 12:50:14.467182 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.467263 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.467288 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.467314 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.467341 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.467375 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.467402 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.467422 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.467446 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.467463 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.470975 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.471013 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.549353 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.549400 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.549453 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.549472 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.549533 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.549666 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.549687 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.549716 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:14.549733 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:14.550054 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.09s) === RUN TestCreateClusterIPv6 I0924 12:50:14.557948 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.558032 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.558058 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.558089 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.558113 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.558147 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.558174 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.558192 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.558214 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.558231 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.561854 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.561895 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.688933 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.689124 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet dualstack-us-test-1a I0924 12:50:14.689145 65034 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0924 12:50:14.689704 65034 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.14s) === RUN TestCreateClusterDifferentAMIs I0924 12:50:14.696251 65034 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0924 12:50:14.696267 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.696345 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.696371 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.696401 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.696426 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.696459 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.696490 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.696510 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.696537 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.696556 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.701083 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.701149 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.745481 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.745568 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:14.745585 65034 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0924 12:50:14.745887 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:14.752712 65034 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.06s) === RUN TestCreateClusterKarpenter I0924 12:50:14.752816 65034 featureflag.go:166] FeatureFlag "Karpenter"=true I0924 12:50:14.752828 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.752908 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.752933 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.752961 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.752985 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.753018 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.753044 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.753064 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.753087 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.753104 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.758264 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.758312 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0924 12:50:14.780561 65034 new_cluster.go:1354] Cloud Provider ID: "aws" I0924 12:50:14.780741 65034 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0924 12:50:14.780757 65034 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0924 12:50:14.780769 65034 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0924 12:50:14.781111 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:14.791144 65034 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.04s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0924 12:50:14.791655 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.791753 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.791788 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.791889 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.791930 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.791980 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.792020 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.792051 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.792089 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.792116 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.800929 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0924 12:50:14.802837 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:14.802952 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:14.802986 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:14.803025 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:14.803064 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:14.803108 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:14.803147 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:14.803179 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:14.803215 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:14.803241 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:14.826713 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:14.829457 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:14.829540 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0924 12:50:17.311239 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:19.889312 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:19.891269 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:19.891303 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:19.891423 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:19.892787 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:19.896398 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:19.896883 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:19.897547 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:19.900013 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:19.902298 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:19.902430 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:19.902804 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:19.903021 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:19.905599 65034 target.go:116] Terraform output is in /tmp/test695068314/out --- PASS: TestMinimal (5.62s) === RUN TestMinimal_v1_23 I0924 12:50:20.424323 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:20.424460 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:20.424505 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:20.424558 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:20.424619 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:20.424671 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:20.424750 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:20.424779 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:20.424818 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:20.424845 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:20.449098 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0924 12:50:20.451983 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:20.452070 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.252680 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:21.380742 65034 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0924 12:50:21.381458 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:21.381494 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:21.381586 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:21.382801 65034 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0924 12:50:21.388797 65034 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0924 12:50:21.389028 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.389220 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.390599 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.390802 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.390867 65034 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0924 12:50:21.391111 65034 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0924 12:50:21.391234 65034 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0924 12:50:21.395176 65034 target.go:116] Terraform output is in /tmp/test4122333353/out --- PASS: TestMinimal_v1_23 (0.97s) === RUN TestMinimal_v1_24 I0924 12:50:21.398197 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:21.398316 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:21.398355 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:21.398402 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:21.398440 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:21.398487 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:21.398565 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:21.398594 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:21.398630 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:21.398655 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:21.422514 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:21.425153 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.425226 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:21.856560 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:21.991676 65034 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0924 12:50:21.992127 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:21.992160 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:21.992248 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:21.998858 65034 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0924 12:50:22.003253 65034 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0924 12:50:22.003557 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.003734 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.004684 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.006413 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.006522 65034 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0924 12:50:22.006764 65034 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0924 12:50:22.006926 65034 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0924 12:50:22.011156 65034 target.go:116] Terraform output is in /tmp/test3267297509/out --- PASS: TestMinimal_v1_24 (0.62s) === RUN TestMinimal_v1_25 I0924 12:50:22.014914 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:22.015053 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:22.015097 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:22.015151 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:22.015196 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:22.015252 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:22.015339 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:22.015376 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:22.015423 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:22.015455 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:22.036201 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:22.038056 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.038114 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.716008 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:22.867452 65034 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0924 12:50:22.868160 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:22.868207 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:22.868309 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:22.869702 65034 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0924 12:50:22.876060 65034 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0924 12:50:22.876452 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.876634 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.878097 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.878155 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.878200 65034 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0924 12:50:22.878450 65034 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0924 12:50:22.878577 65034 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0924 12:50:22.882893 65034 target.go:116] Terraform output is in /tmp/test3639599128/out --- PASS: TestMinimal_v1_25 (0.87s) === RUN TestMinimal_v1_26 I0924 12:50:22.886185 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:22.886298 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:22.886334 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:22.886377 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:22.886418 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:22.886467 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:22.886545 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:22.886574 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:22.886611 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:22.886638 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:22.911141 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:22.914315 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:22.914394 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:23.667202 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:23.829346 65034 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0924 12:50:23.833194 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:23.833278 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:23.833455 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:23.835791 65034 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0924 12:50:23.838905 65034 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0924 12:50:23.839250 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:23.839449 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:23.840468 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:23.841877 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:23.841975 65034 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0924 12:50:23.842199 65034 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0924 12:50:23.842342 65034 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0924 12:50:23.846293 65034 target.go:116] Terraform output is in /tmp/test1737113609/out --- PASS: TestMinimal_v1_26 (0.96s) === RUN TestMinimal_NoneDNS I0924 12:50:23.849381 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:23.849495 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:23.849532 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:23.849571 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:23.849607 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:23.849661 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:23.849734 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:23.849762 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:23.849798 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:23.849829 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:23.877028 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:23.877111 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.085823 65034 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0924 12:50:24.092118 65034 executor.go:111] Tasks: 43 done / 98 total; 20 can run I0924 12:50:24.094483 65034 executor.go:111] Tasks: 63 done / 98 total; 25 can run I0924 12:50:24.095101 65034 executor.go:111] Tasks: 88 done / 98 total; 2 can run I0924 12:50:24.095152 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:24.095234 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:24.095337 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:24.095398 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:24.100260 65034 executor.go:111] Tasks: 90 done / 98 total; 4 can run I0924 12:50:24.100637 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.102298 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.102832 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.105866 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.106154 65034 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0924 12:50:24.106816 65034 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0924 12:50:24.107154 65034 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0924 12:50:24.119827 65034 target.go:116] Terraform output is in /tmp/test2315792635/out --- PASS: TestMinimal_NoneDNS (0.27s) === RUN TestHetzner I0924 12:50:24.122431 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:24.836271 65034 executor.go:111] Tasks: 0 done / 46 total; 37 can run I0924 12:50:24.842529 65034 executor.go:111] Tasks: 37 done / 46 total; 3 can run I0924 12:50:24.842834 65034 executor.go:111] Tasks: 40 done / 46 total; 2 can run I0924 12:50:24.845597 65034 executor.go:111] Tasks: 42 done / 46 total; 4 can run I0924 12:50:24.848125 65034 executor.go:111] Tasks: 46 done / 46 total; 0 can run I0924 12:50:24.850141 65034 target.go:116] Terraform output is in /tmp/test2037151056/out --- PASS: TestHetzner (0.73s) === RUN TestNvidia I0924 12:50:24.852569 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:24.852676 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:24.852711 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:24.852753 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:24.852793 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:24.852840 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:24.852916 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:24.852947 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:24.852985 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:24.853012 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:24.870265 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:24.871933 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.871989 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:24.949812 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:25.031502 65034 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0924 12:50:25.031973 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:25.032011 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:25.032083 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:25.033615 65034 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0924 12:50:25.036536 65034 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0924 12:50:25.036670 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.036764 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.037567 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.038145 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.038216 65034 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0924 12:50:25.038489 65034 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0924 12:50:25.038622 65034 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0924 12:50:25.043076 65034 target.go:116] Terraform output is in /tmp/test1817621428/out --- PASS: TestNvidia (0.19s) === RUN TestMinimalGossip I0924 12:50:25.046171 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:25.046283 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:25.046318 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:25.046363 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:25.046402 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:25.046452 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:25.046492 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:25.046521 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:25.046559 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:25.046588 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:25.067108 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.067228 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.217699 65034 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0924 12:50:25.219720 65034 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0924 12:50:25.222588 65034 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0924 12:50:25.222759 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.222959 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.223539 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.224257 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.224320 65034 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0924 12:50:25.224488 65034 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0924 12:50:25.224579 65034 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0924 12:50:25.227308 65034 target.go:116] Terraform output is in /tmp/test879399785/out --- PASS: TestMinimalGossip (0.18s) === RUN TestMinimalGossipIRSA I0924 12:50:25.229537 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:25.229629 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:25.229659 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:25.229691 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:25.229715 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:25.229750 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:25.229777 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:25.229794 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:25.229820 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:25.229838 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:25.256284 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.256368 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.431386 65034 executor.go:111] Tasks: 0 done / 89 total; 44 can run I0924 12:50:25.434281 65034 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0924 12:50:25.440222 65034 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0924 12:50:25.440447 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.440663 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.441512 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.442983 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:25.443086 65034 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0924 12:50:25.443326 65034 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0924 12:50:25.443518 65034 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0924 12:50:25.448191 65034 target.go:116] Terraform output is in /tmp/test834402672/out --- PASS: TestMinimalGossipIRSA (0.22s) === RUN TestMinimalGCE I0924 12:50:25.451832 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:25.453632 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:26.577400 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:26.577469 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:26.577565 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0924 12:50:26.577590 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0924 12:50:26.577605 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0924 12:50:26.577653 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:26.578567 65034 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0924 12:50:26.584363 65034 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0924 12:50:26.587478 65034 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0924 12:50:26.590289 65034 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0924 12:50:26.590782 65034 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0924 12:50:26.592672 65034 target.go:116] Terraform output is in /tmp/test1207572581/out --- PASS: TestMinimalGCE (1.14s) === RUN TestMinimalGCEPrivate I0924 12:50:26.595053 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:26.596268 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:26.831905 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:26.831999 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:26.832129 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0924 12:50:26.832160 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0924 12:50:26.832185 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0924 12:50:26.832258 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:26.833725 65034 executor.go:111] Tasks: 0 done / 65 total; 41 can run I0924 12:50:26.835740 65034 executor.go:111] Tasks: 41 done / 65 total; 17 can run I0924 12:50:26.839970 65034 executor.go:111] Tasks: 58 done / 65 total; 5 can run I0924 12:50:26.842412 65034 executor.go:111] Tasks: 63 done / 65 total; 2 can run I0924 12:50:26.842559 65034 executor.go:111] Tasks: 65 done / 65 total; 0 can run I0924 12:50:26.845351 65034 target.go:116] Terraform output is in /tmp/test1628017411/out --- PASS: TestMinimalGCEPrivate (0.25s) === RUN TestMinimalGCEInternalLoadBalancer I0924 12:50:26.848121 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:26.849767 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:27.123795 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:27.123888 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:27.124024 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0924 12:50:27.124057 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0924 12:50:27.124084 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0924 12:50:27.124158 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0924 12:50:27.124205 65034 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0924 12:50:27.125336 65034 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0924 12:50:27.131667 65034 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0924 12:50:27.134727 65034 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0924 12:50:27.134751 65034 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0924 12:50:27.136592 65034 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0924 12:50:27.136744 65034 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0924 12:50:27.136940 65034 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0924 12:50:27.137089 65034 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0924 12:50:27.139057 65034 target.go:116] Terraform output is in /tmp/test3127678047/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.29s) === RUN TestMinimalGCEPublicLoadBalancer I0924 12:50:27.141015 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:27.142176 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:27.364476 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:27.364529 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:27.364635 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/main" W0924 12:50:27.364666 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com/backups/etcd/events" W0924 12:50:27.364684 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-plb.example.com" W0924 12:50:27.364741 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:27.365688 65034 executor.go:111] Tasks: 0 done / 70 total; 43 can run I0924 12:50:27.367811 65034 executor.go:111] Tasks: 43 done / 70 total; 18 can run I0924 12:50:27.370670 65034 executor.go:111] Tasks: 61 done / 70 total; 7 can run I0924 12:50:27.372341 65034 executor.go:111] Tasks: 68 done / 70 total; 2 can run I0924 12:50:27.372516 65034 executor.go:111] Tasks: 70 done / 70 total; 0 can run I0924 12:50:27.375297 65034 target.go:116] Terraform output is in /tmp/test2279806528/out --- PASS: TestMinimalGCEPublicLoadBalancer (0.24s) === RUN TestMinimalGCELongClusterName I0924 12:50:27.378054 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:27.379647 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:27.601915 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:27.601999 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:27.602098 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0924 12:50:27.602121 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0924 12:50:27.602137 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0924 12:50:27.602190 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:27.603080 65034 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0924 12:50:27.608600 65034 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0924 12:50:27.611735 65034 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0924 12:50:27.614318 65034 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0924 12:50:27.614470 65034 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0924 12:50:27.617289 65034 target.go:116] Terraform output is in /tmp/test2549643955/out --- PASS: TestMinimalGCELongClusterName (0.24s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0924 12:50:27.620089 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:27.621832 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:27.858677 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:27.858768 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:27.858927 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0924 12:50:27.858959 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0924 12:50:27.858985 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0924 12:50:27.859061 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0924 12:50:27.859110 65034 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0924 12:50:27.860605 65034 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0924 12:50:27.862740 65034 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0924 12:50:27.865673 65034 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0924 12:50:27.865695 65034 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0924 12:50:27.868208 65034 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0924 12:50:27.868392 65034 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0924 12:50:27.868491 65034 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0924 12:50:27.868584 65034 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0924 12:50:27.870459 65034 target.go:116] Terraform output is in /tmp/test808556230/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.25s) === RUN TestMinimalGCEDNSNone I0924 12:50:27.872429 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:27.873567 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:28.078286 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:28.078346 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:28.078434 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0924 12:50:28.078456 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0924 12:50:28.078471 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0924 12:50:28.078506 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0924 12:50:28.078535 65034 autoscalinggroup.go:296] Not hooking the instance group manager up to anything. I0924 12:50:28.079428 65034 executor.go:111] Tasks: 0 done / 67 total; 41 can run I0924 12:50:28.081792 65034 executor.go:111] Tasks: 41 done / 67 total; 16 can run W0924 12:50:28.088093 65034 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0924 12:50:28.088130 65034 executor.go:111] Tasks: 57 done / 67 total; 5 can run I0924 12:50:28.090879 65034 executor.go:111] Tasks: 62 done / 67 total; 2 can run I0924 12:50:28.091062 65034 executor.go:111] Tasks: 64 done / 67 total; 1 can run I0924 12:50:28.091167 65034 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0924 12:50:28.091340 65034 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0924 12:50:28.094209 65034 target.go:116] Terraform output is in /tmp/test1909422198/out --- PASS: TestMinimalGCEDNSNone (0.22s) === RUN TestHA I0924 12:50:28.096971 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:28.097084 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:28.097120 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:28.097167 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:28.097211 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:28.097258 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:28.097301 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:28.097334 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:28.097373 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:28.097400 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:28.128336 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:28.131213 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.131300 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.131353 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.131402 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.202981 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:28.269772 65034 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0924 12:50:28.270117 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:28.270177 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:28.270280 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:28.275476 65034 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0924 12:50:28.280440 65034 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0924 12:50:28.280616 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.280889 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.281066 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.281260 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.281918 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.282060 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.282343 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.282612 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.282687 65034 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0924 12:50:28.282960 65034 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0924 12:50:28.283113 65034 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0924 12:50:28.286843 65034 target.go:116] Terraform output is in /tmp/test1006624488/out --- PASS: TestHA (0.20s) === RUN TestHighAvailabilityGCE I0924 12:50:28.292294 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:28.293599 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:28.566712 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:28.566823 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:28.566952 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0924 12:50:28.566987 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0924 12:50:28.567013 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0924 12:50:28.567111 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0924 12:50:28.567198 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups W0924 12:50:28.567274 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:28.569202 65034 executor.go:111] Tasks: 0 done / 82 total; 49 can run I0924 12:50:28.571901 65034 executor.go:111] Tasks: 49 done / 82 total; 19 can run I0924 12:50:28.576817 65034 executor.go:111] Tasks: 68 done / 82 total; 8 can run I0924 12:50:28.584903 65034 executor.go:111] Tasks: 76 done / 82 total; 6 can run I0924 12:50:28.585303 65034 executor.go:111] Tasks: 82 done / 82 total; 0 can run I0924 12:50:28.589807 65034 target.go:116] Terraform output is in /tmp/test2805515022/out --- PASS: TestHighAvailabilityGCE (0.30s) === RUN TestComplex I0924 12:50:28.594067 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:28.594190 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:28.594234 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:28.594326 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:28.594372 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:28.594426 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:28.594469 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:28.594503 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:28.594544 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:28.594575 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:28.626937 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:28.630804 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:28.630897 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.255185 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:29.383363 65034 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0924 12:50:29.388010 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:29.388082 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:29.388412 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:29.389688 65034 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0924 12:50:29.393816 65034 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0924 12:50:29.394477 65034 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0924 12:50:29.394536 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:29.394625 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:29.394733 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:29.394805 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:29.399104 65034 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0924 12:50:29.399326 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.399548 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.401064 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.402341 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.402467 65034 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0924 12:50:29.402774 65034 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0924 12:50:29.402931 65034 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0924 12:50:29.408423 65034 target.go:116] Terraform output is in /tmp/test2867130997/out I0924 12:50:29.411862 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:29.411956 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:29.411997 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:29.412044 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:29.412084 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:29.412133 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:29.412174 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:29.412204 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:29.412254 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:29.412286 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:29.440737 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:29.444113 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.444196 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.518626 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:29.651176 65034 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0924 12:50:29.651838 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:29.651871 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:29.651935 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:29.653304 65034 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0924 12:50:29.658507 65034 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0924 12:50:29.659163 65034 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0924 12:50:29.659217 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:29.659299 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:29.659423 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:29.659495 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:29.664525 65034 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0924 12:50:29.664760 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.665005 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.666344 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.666825 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.666908 65034 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0924 12:50:29.667224 65034 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0924 12:50:29.667376 65034 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0924 12:50:29.673387 65034 target.go:116] Terraform output is in /tmp/test3965067311/out --- PASS: TestComplex (1.08s) === RUN TestCompress I0924 12:50:29.677213 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:29.677341 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:29.677383 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:29.677431 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:29.677474 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:29.677527 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:29.677571 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:29.677602 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:29.677645 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:29.677675 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:29.701348 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:29.704198 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.704285 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.776295 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:29.861501 65034 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0924 12:50:29.861883 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:29.861911 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:29.862000 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:29.863582 65034 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0924 12:50:29.866428 65034 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0924 12:50:29.866707 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.866855 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.869728 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.872778 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.872878 65034 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0924 12:50:29.873083 65034 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0924 12:50:29.873239 65034 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0924 12:50:29.875711 65034 target.go:116] Terraform output is in /tmp/test1368203000/out --- PASS: TestCompress (0.20s) === RUN TestExternalPolicies I0924 12:50:29.877628 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:29.877705 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:29.877729 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:29.877760 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:29.877786 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:29.877819 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:29.877847 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:29.877864 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:29.877889 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:29.877908 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:29.894042 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:29.896043 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.896106 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:29.969582 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:30.039536 65034 executor.go:111] Tasks: 0 done / 100 total; 45 can run I0924 12:50:30.041897 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:30.041921 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:30.041978 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:30.045568 65034 executor.go:111] Tasks: 45 done / 100 total; 18 can run I0924 12:50:30.047367 65034 executor.go:111] Tasks: 63 done / 100 total; 26 can run I0924 12:50:30.047872 65034 executor.go:111] Tasks: 89 done / 100 total; 3 can run W0924 12:50:30.047992 65034 api.go:53] PageSize not implemented W0924 12:50:30.048105 65034 api.go:53] PageSize not implemented I0924 12:50:30.050697 65034 executor.go:111] Tasks: 92 done / 100 total; 4 can run I0924 12:50:30.050821 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.050985 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.051691 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.052368 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.052438 65034 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0924 12:50:30.052609 65034 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0924 12:50:30.052751 65034 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0924 12:50:30.055779 65034 target.go:116] Terraform output is in /tmp/test946386850/out --- PASS: TestExternalPolicies (0.18s) === RUN TestMinimalIPv6 I0924 12:50:30.057963 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:30.058043 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:30.058069 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:30.058096 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:30.058124 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:30.058158 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:30.058207 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:30.058229 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:30.058254 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:30.058272 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:30.075021 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:30.077065 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.077123 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.152166 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:30.260971 65034 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0924 12:50:30.261466 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:30.261500 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:30.261593 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:30.263310 65034 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0924 12:50:30.265153 65034 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0924 12:50:30.265641 65034 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0924 12:50:30.265968 65034 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0924 12:50:30.266058 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:30.266110 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:30.266223 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:30.266277 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:30.271220 65034 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0924 12:50:30.271342 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.271499 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.272908 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.272997 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.273092 65034 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0924 12:50:30.273310 65034 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0924 12:50:30.273427 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:30.280695 65034 target.go:116] Terraform output is in /tmp/test2576601908/out --- PASS: TestMinimalIPv6 (0.22s) === RUN TestMinimalIPv6Calico I0924 12:50:30.282919 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:30.282995 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:30.283021 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:30.283052 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:30.283079 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:30.283113 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:30.283164 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:30.283182 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:30.283205 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:30.283222 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:30.300881 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:30.302994 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.303054 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.373658 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:30.666534 65034 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0924 12:50:30.667108 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:30.667155 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:30.667244 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:30.670116 65034 executor.go:111] Tasks: 47 done / 126 total; 21 can run I0924 12:50:30.673484 65034 executor.go:111] Tasks: 68 done / 126 total; 33 can run I0924 12:50:30.673965 65034 executor.go:111] Tasks: 101 done / 126 total; 9 can run I0924 12:50:30.674252 65034 executor.go:111] Tasks: 110 done / 126 total; 8 can run I0924 12:50:30.674307 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:30.674376 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:30.674488 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:30.674542 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:30.678815 65034 executor.go:111] Tasks: 118 done / 126 total; 4 can run I0924 12:50:30.678938 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.679109 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.679846 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.681497 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.681598 65034 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0924 12:50:30.681802 65034 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0924 12:50:30.681949 65034 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0924 12:50:30.685486 65034 target.go:116] Terraform output is in /tmp/test915771740/out --- PASS: TestMinimalIPv6Calico (0.42s) === RUN TestMinimalIPv6Cilium I0924 12:50:30.700118 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:30.700206 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:30.700237 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:30.700267 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:30.700296 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:30.700329 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:30.700384 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:30.700402 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:30.700426 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:30.700444 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:30.717820 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:30.719990 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.720046 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.791932 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:30.938721 65034 executor.go:111] Tasks: 0 done / 125 total; 47 can run I0924 12:50:30.939145 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:30.939166 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:30.939240 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:30.940940 65034 executor.go:111] Tasks: 47 done / 125 total; 21 can run I0924 12:50:30.942889 65034 executor.go:111] Tasks: 68 done / 125 total; 32 can run I0924 12:50:30.943385 65034 executor.go:111] Tasks: 100 done / 125 total; 9 can run I0924 12:50:30.943691 65034 executor.go:111] Tasks: 109 done / 125 total; 8 can run I0924 12:50:30.943738 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:30.943791 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:30.943852 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:30.943889 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:30.946485 65034 executor.go:111] Tasks: 117 done / 125 total; 4 can run I0924 12:50:30.947197 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.947416 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.950864 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.951366 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.951445 65034 executor.go:111] Tasks: 121 done / 125 total; 2 can run I0924 12:50:30.951645 65034 executor.go:111] Tasks: 123 done / 125 total; 2 can run I0924 12:50:30.951763 65034 executor.go:111] Tasks: 125 done / 125 total; 0 can run I0924 12:50:30.955285 65034 target.go:116] Terraform output is in /tmp/test2891314075/out --- PASS: TestMinimalIPv6Cilium (0.26s) === RUN TestMinimalIPv6NoSubnetPrefix I0924 12:50:30.957648 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:30.957729 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:30.957755 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:30.957784 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:30.957808 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:30.957840 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:30.957893 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:30.957911 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:30.957945 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:30.957963 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:30.974937 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:30.977150 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:30.977209 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.048592 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:31.186418 65034 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0924 12:50:31.187242 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:31.187281 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:31.187386 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:31.189614 65034 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0924 12:50:31.195160 65034 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0924 12:50:31.196050 65034 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0924 12:50:31.196435 65034 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0924 12:50:31.196592 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:31.196681 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:31.196790 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:31.196863 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:31.199768 65034 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0924 12:50:31.199906 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.200033 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.200874 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.202943 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.203069 65034 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0924 12:50:31.203331 65034 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0924 12:50:31.203562 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:31.209703 65034 target.go:116] Terraform output is in /tmp/test2722233307/out --- PASS: TestMinimalIPv6NoSubnetPrefix (0.26s) === RUN TestMinimalWarmPool I0924 12:50:31.213726 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:31.213878 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:31.213922 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:31.213976 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:31.214031 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:31.214099 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:31.214203 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:31.214243 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:31.214300 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:31.214338 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:31.239405 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:31.242859 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.242949 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.316275 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:31.494831 65034 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0924 12:50:31.495504 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:31.495539 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:31.495603 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:31.497400 65034 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0924 12:50:31.500597 65034 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0924 12:50:31.500804 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.501017 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.501933 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.502425 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.502511 65034 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0924 12:50:31.502685 65034 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0924 12:50:31.502801 65034 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0924 12:50:31.505574 65034 target.go:116] Terraform output is in /tmp/test279084423/out --- PASS: TestMinimalWarmPool (0.29s) === RUN TestMinimalEtcd I0924 12:50:31.507803 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:31.507890 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:31.507930 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:31.507985 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:31.508030 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:31.508083 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:31.508119 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:31.508150 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:31.508190 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:31.508216 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:31.535001 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:31.538005 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.538106 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.610232 65034 zones.go:93] ListHostedZonesPages { } W0924 12:50:31.680334 65034 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0924 12:50:31.680364 65034 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 10 W0924 12:50:31.681396 65034 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0924 12:50:31.688275 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:31.688806 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:31.688855 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:31.688956 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:31.691047 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:31.697950 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:31.698328 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.698556 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.699857 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.701171 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.701306 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:31.701592 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:31.701776 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:31.706024 65034 target.go:116] Terraform output is in /tmp/test2853394997/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0924 12:50:31.709589 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:31.709714 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:31.709754 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:31.709798 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:31.709839 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:31.709891 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:31.709937 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:31.709968 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:31.710009 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:31.710038 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:31.735209 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:31.738118 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.738204 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.810239 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:31.898433 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:31.898980 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:31.899003 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:31.899063 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:31.900806 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:31.903708 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:31.904026 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.904170 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.904937 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.905598 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.905659 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:31.906036 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:31.906217 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:31.908735 65034 target.go:116] Terraform output is in /tmp/test3088895613/out --- PASS: TestMinimalGp3 (0.20s) === RUN TestMinimalLongClusterName I0924 12:50:31.910684 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:31.910765 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:31.910789 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:31.910816 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:31.910838 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:31.910870 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:31.910895 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:31.910912 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:31.910936 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:31.910955 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:31.924895 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:31.926538 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:31.926591 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0924 12:50:31.997456 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:32.091558 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:32.092214 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:32.092250 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:32.092341 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:32.093986 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:32.096868 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:32.097301 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.097526 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.098728 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.099969 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.100088 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:32.100371 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:32.100555 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:32.105077 65034 target.go:116] Terraform output is in /tmp/test678542847/out --- PASS: TestMinimalLongClusterName (0.20s) === RUN TestExistingSG I0924 12:50:32.108597 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:32.108727 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:32.108766 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:32.108816 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:32.108864 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:32.108918 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:32.108961 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:32.108993 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:32.109031 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:32.109060 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:32.136118 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:32.138063 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.138121 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.138155 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.138186 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.211129 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:32.295489 65034 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0924 12:50:32.296112 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:32.296137 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:32.296222 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:32.298202 65034 executor.go:111] Tasks: 53 done / 146 total; 22 can run I0924 12:50:32.300212 65034 executor.go:111] Tasks: 75 done / 146 total; 50 can run I0924 12:50:32.300954 65034 executor.go:111] Tasks: 125 done / 146 total; 5 can run W0924 12:50:32.301013 65034 api.go:53] PageSize not implemented W0924 12:50:32.301094 65034 api.go:53] PageSize not implemented W0924 12:50:32.301154 65034 api.go:53] PageSize not implemented W0924 12:50:32.301236 65034 api.go:53] PageSize not implemented I0924 12:50:32.305798 65034 executor.go:111] Tasks: 130 done / 146 total; 8 can run I0924 12:50:32.306006 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.306280 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.306519 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.306805 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.307631 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.308685 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.309916 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.310361 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.310491 65034 executor.go:111] Tasks: 138 done / 146 total; 4 can run I0924 12:50:32.310884 65034 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0924 12:50:32.311103 65034 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0924 12:50:32.322532 65034 target.go:116] Terraform output is in /tmp/test88782747/out --- PASS: TestExistingSG (0.22s) === RUN TestBastionAdditionalUserData I0924 12:50:32.327185 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:32.327321 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:32.327362 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:32.327410 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:32.327455 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:32.327510 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:32.327553 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:32.327583 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:32.327621 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:32.327650 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:32.357463 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:32.360609 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.360704 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.360769 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.433049 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:32.509686 65034 executor.go:111] Tasks: 0 done / 121 total; 46 can run I0924 12:50:32.510425 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:32.510449 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:32.510512 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:32.513417 65034 executor.go:111] Tasks: 46 done / 121 total; 26 can run I0924 12:50:32.518945 65034 executor.go:111] Tasks: 72 done / 121 total; 31 can run I0924 12:50:32.519791 65034 executor.go:111] Tasks: 103 done / 121 total; 5 can run W0924 12:50:32.519905 65034 api.go:53] PageSize not implemented W0924 12:50:32.520041 65034 api.go:53] PageSize not implemented W0924 12:50:32.520150 65034 api.go:53] PageSize not implemented I0924 12:50:32.524725 65034 executor.go:111] Tasks: 108 done / 121 total; 7 can run I0924 12:50:32.524910 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.525073 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.525364 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.525867 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.526425 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.527773 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.527889 65034 executor.go:111] Tasks: 115 done / 121 total; 3 can run I0924 12:50:32.528207 65034 executor.go:111] Tasks: 118 done / 121 total; 3 can run I0924 12:50:32.528463 65034 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0924 12:50:32.534484 65034 target.go:116] Terraform output is in /tmp/test1251827452/out --- PASS: TestBastionAdditionalUserData (0.21s) === RUN TestPrivateWeave I0924 12:50:32.538486 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:32.538611 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:32.538652 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:32.538696 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:32.538737 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:32.538796 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:32.538878 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:32.538909 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:32.539299 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:32.539334 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:32.567117 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:32.570154 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.570234 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.570294 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.642222 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:32.732938 65034 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0924 12:50:32.733659 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:32.733693 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:32.733785 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:32.735613 65034 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0924 12:50:32.739030 65034 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0924 12:50:32.739542 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.739699 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.739798 65034 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0924 12:50:32.739879 65034 api.go:53] PageSize not implemented W0924 12:50:32.740012 65034 api.go:53] PageSize not implemented I0924 12:50:32.744451 65034 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0924 12:50:32.744624 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.744814 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.747739 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.750939 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.751084 65034 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0924 12:50:32.751365 65034 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0924 12:50:32.751505 65034 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0924 12:50:32.757775 65034 target.go:116] Terraform output is in /tmp/test477024105/out --- PASS: TestPrivateWeave (0.22s) === RUN TestPrivateFlannel I0924 12:50:32.761995 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:32.762118 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:32.762158 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:32.762249 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:32.762292 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:32.762343 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:32.762386 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:32.762417 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:32.762455 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:32.762485 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:32.790244 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:32.793262 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.793337 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.793395 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:32.864305 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:33.003014 65034 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0924 12:50:33.003918 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:33.003948 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:33.004015 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:33.006562 65034 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0924 12:50:33.009854 65034 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0924 12:50:33.010261 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.010351 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.010663 65034 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0924 12:50:33.010758 65034 api.go:53] PageSize not implemented W0924 12:50:33.010815 65034 api.go:53] PageSize not implemented I0924 12:50:33.015735 65034 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0924 12:50:33.015922 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.016211 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.017710 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.018644 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.018765 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:50:33.019070 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:50:33.019284 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:50:33.026215 65034 target.go:116] Terraform output is in /tmp/test1306664689/out --- PASS: TestPrivateFlannel (0.27s) === RUN TestPrivateCalico I0924 12:50:33.034643 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:33.034784 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:33.034828 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:33.034881 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:33.034923 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:33.034974 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:33.035071 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:33.035105 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:33.035146 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:33.035175 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:33.064807 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:33.068252 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.068337 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.068398 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.140252 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:33.494846 65034 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0924 12:50:33.495175 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:33.495199 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:33.495263 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:33.497320 65034 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0924 12:50:33.500258 65034 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0924 12:50:33.500754 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.500852 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.500920 65034 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0924 12:50:33.500989 65034 api.go:53] PageSize not implemented W0924 12:50:33.501132 65034 api.go:53] PageSize not implemented I0924 12:50:33.505878 65034 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0924 12:50:33.506087 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.506344 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.507231 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.508753 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.508878 65034 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0924 12:50:33.509149 65034 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0924 12:50:33.509327 65034 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0924 12:50:33.513160 65034 target.go:116] Terraform output is in /tmp/test3703150628/out --- PASS: TestPrivateCalico (0.48s) === RUN TestPrivateCilium I0924 12:50:33.519626 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:33.519788 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:33.519831 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:33.519883 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:33.519926 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:33.519979 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:33.520065 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:33.520097 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:33.520138 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:33.520168 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:33.541943 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:33.544049 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.544119 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.544177 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.615206 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:33.735295 65034 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0924 12:50:33.736289 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:33.736353 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:33.736496 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:33.737985 65034 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0924 12:50:33.740970 65034 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0924 12:50:33.741456 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.741607 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.741709 65034 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0924 12:50:33.741771 65034 api.go:53] PageSize not implemented W0924 12:50:33.741807 65034 api.go:53] PageSize not implemented I0924 12:50:33.744540 65034 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0924 12:50:33.744777 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.744949 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.746152 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.746424 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.746543 65034 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0924 12:50:33.746776 65034 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0924 12:50:33.746952 65034 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0924 12:50:33.750509 65034 target.go:116] Terraform output is in /tmp/test554122247/out --- PASS: TestPrivateCilium (0.23s) === RUN TestPrivateCilium2 I0924 12:50:33.752738 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:33.752812 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:33.752837 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:33.752867 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:33.752891 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:33.752923 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:33.752973 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:33.752991 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:33.753015 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:33.753031 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:33.773284 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:33.775445 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.775505 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.775540 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:33.848287 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:34.433206 65034 executor.go:111] Tasks: 0 done / 124 total; 51 can run I0924 12:50:34.434033 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:34.434063 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:34.434158 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:34.435859 65034 executor.go:111] Tasks: 51 done / 124 total; 26 can run I0924 12:50:34.441767 65034 executor.go:111] Tasks: 77 done / 124 total; 32 can run I0924 12:50:34.442140 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.442388 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.442501 65034 executor.go:111] Tasks: 109 done / 124 total; 5 can run W0924 12:50:34.442602 65034 api.go:53] PageSize not implemented W0924 12:50:34.442741 65034 api.go:53] PageSize not implemented I0924 12:50:34.447840 65034 executor.go:111] Tasks: 114 done / 124 total; 6 can run I0924 12:50:34.448080 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.448304 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.449648 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.451097 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.451212 65034 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0924 12:50:34.451504 65034 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0924 12:50:34.451744 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:34.458647 65034 target.go:116] Terraform output is in /tmp/test3724024598/out --- PASS: TestPrivateCilium2 (0.71s) === RUN TestPrivateCiliumAdvanced I0924 12:50:34.466884 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:34.467011 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:34.467052 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:34.467095 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:34.467142 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:34.467194 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:34.467303 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:34.467334 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:34.467376 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:34.467408 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:34.498624 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:34.501410 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.501476 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.501525 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.574103 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:34.709765 65034 executor.go:111] Tasks: 0 done / 126 total; 53 can run I0924 12:50:34.710265 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:34.710309 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:34.710414 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:34.717200 65034 executor.go:111] Tasks: 53 done / 126 total; 26 can run I0924 12:50:34.720623 65034 executor.go:111] Tasks: 79 done / 126 total; 32 can run I0924 12:50:34.721117 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.721248 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.721327 65034 executor.go:111] Tasks: 111 done / 126 total; 5 can run W0924 12:50:34.721422 65034 api.go:53] PageSize not implemented W0924 12:50:34.721517 65034 api.go:53] PageSize not implemented I0924 12:50:34.727253 65034 executor.go:111] Tasks: 116 done / 126 total; 6 can run I0924 12:50:34.727418 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.727672 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.729038 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.729799 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.729913 65034 executor.go:111] Tasks: 122 done / 126 total; 2 can run I0924 12:50:34.730189 65034 executor.go:111] Tasks: 124 done / 126 total; 2 can run I0924 12:50:34.730352 65034 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0924 12:50:34.736709 65034 target.go:116] Terraform output is in /tmp/test1194308215/out --- PASS: TestPrivateCiliumAdvanced (0.27s) === RUN TestPrivateCiliumENI I0924 12:50:34.740838 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:34.740971 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:34.741011 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:34.741058 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:34.741099 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:34.741154 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:34.741235 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:34.741312 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:34.741352 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:34.741389 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:34.769533 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:34.772551 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.772624 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.772679 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.845765 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:34.977479 65034 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0924 12:50:34.978208 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:34.978251 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:34.978333 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:34.979139 65034 executor.go:111] Tasks: 47 done / 120 total; 26 can run I0924 12:50:34.981043 65034 executor.go:111] Tasks: 73 done / 120 total; 32 can run I0924 12:50:34.981176 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.981282 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.981556 65034 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0924 12:50:34.981691 65034 api.go:53] PageSize not implemented W0924 12:50:34.981793 65034 api.go:53] PageSize not implemented I0924 12:50:34.985608 65034 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0924 12:50:34.985725 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.985817 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.988121 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.990951 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:34.991033 65034 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0924 12:50:34.991325 65034 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0924 12:50:34.991491 65034 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0924 12:50:34.995263 65034 target.go:116] Terraform output is in /tmp/test355561903/out --- PASS: TestPrivateCiliumENI (0.26s) === RUN TestPrivateCanal I0924 12:50:34.997477 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:34.997554 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:34.997582 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:34.997643 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:34.997669 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:34.997707 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:34.997736 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:34.997754 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:34.997778 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:34.997797 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:35.013875 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:35.015781 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.015838 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.015873 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.088379 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:35.507619 65034 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0924 12:50:35.508291 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:35.508321 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:35.508427 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:35.510231 65034 executor.go:111] Tasks: 50 done / 123 total; 26 can run I0924 12:50:35.512252 65034 executor.go:111] Tasks: 76 done / 123 total; 32 can run I0924 12:50:35.512712 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.512934 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.513053 65034 executor.go:111] Tasks: 108 done / 123 total; 5 can run W0924 12:50:35.513152 65034 api.go:53] PageSize not implemented W0924 12:50:35.513226 65034 api.go:53] PageSize not implemented I0924 12:50:35.518280 65034 executor.go:111] Tasks: 113 done / 123 total; 6 can run I0924 12:50:35.518510 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.518670 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.520024 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.520231 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.520342 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:50:35.520593 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:50:35.520724 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:50:35.526988 65034 target.go:116] Terraform output is in /tmp/test4038032448/out --- PASS: TestPrivateCanal (0.54s) === RUN TestPrivateKopeio I0924 12:50:35.533886 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:35.533976 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:35.534002 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:35.534034 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:35.534059 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:35.534091 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:35.534142 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:35.534160 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:35.534184 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:35.534202 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:35.561285 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:35.564454 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.564543 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.564730 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.636887 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:35.734273 65034 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0924 12:50:35.734786 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:35.734820 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:35.734896 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:35.741402 65034 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0924 12:50:35.744930 65034 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0924 12:50:35.745623 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.745783 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.745897 65034 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0924 12:50:35.746091 65034 api.go:53] PageSize not implemented W0924 12:50:35.746225 65034 api.go:53] PageSize not implemented I0924 12:50:35.750893 65034 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0924 12:50:35.751083 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.751374 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.753617 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.754520 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.754666 65034 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0924 12:50:35.755085 65034 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0924 12:50:35.755286 65034 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0924 12:50:35.761523 65034 target.go:116] Terraform output is in /tmp/test3640581659/out --- PASS: TestPrivateKopeio (0.24s) === RUN TestUnmanaged I0924 12:50:35.775504 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:35.775639 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:35.775684 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:35.775731 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:35.775779 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:35.775834 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:35.775904 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:35.775935 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:35.775977 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:35.776010 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:35.807835 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:35.812103 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.812170 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.812207 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.884700 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:35.973269 65034 executor.go:111] Tasks: 0 done / 110 total; 45 can run I0924 12:50:35.975156 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:35.975216 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:35.975294 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:35.980028 65034 executor.go:111] Tasks: 45 done / 110 total; 23 can run I0924 12:50:35.983305 65034 executor.go:111] Tasks: 68 done / 110 total; 29 can run I0924 12:50:35.983929 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.984087 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.984189 65034 executor.go:111] Tasks: 97 done / 110 total; 4 can run W0924 12:50:35.984235 65034 api.go:53] PageSize not implemented W0924 12:50:35.984439 65034 api.go:53] PageSize not implemented I0924 12:50:35.989052 65034 executor.go:111] Tasks: 101 done / 110 total; 5 can run I0924 12:50:35.989285 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.989513 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.990978 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.992040 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:35.992240 65034 executor.go:111] Tasks: 106 done / 110 total; 2 can run I0924 12:50:35.992543 65034 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0924 12:50:35.992722 65034 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0924 12:50:35.998276 65034 target.go:116] Terraform output is in /tmp/test2032784378/out --- PASS: TestUnmanaged (0.23s) === RUN TestPrivateSharedSubnet I0924 12:50:36.002057 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:36.002237 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:36.002299 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:36.002364 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:36.002410 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:36.002463 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:36.002505 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:36.002536 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:36.002575 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:36.002604 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:36.031658 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:36.034939 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.035026 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.035085 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.108616 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:36.178865 65034 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0924 12:50:36.179644 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:36.179668 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:36.179729 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:36.182446 65034 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0924 12:50:36.182880 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:36.185804 65034 executor.go:111] Tasks: 67 done / 107 total; 27 can run I0924 12:50:36.186253 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.186466 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.186585 65034 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0924 12:50:36.186642 65034 api.go:53] PageSize not implemented W0924 12:50:36.186815 65034 api.go:53] PageSize not implemented I0924 12:50:36.191488 65034 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0924 12:50:36.191791 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.192033 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.194152 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.195333 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.195464 65034 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0924 12:50:36.195820 65034 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0924 12:50:36.196025 65034 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0924 12:50:36.203257 65034 target.go:116] Terraform output is in /tmp/test2330972677/out --- PASS: TestPrivateSharedSubnet (0.20s) === RUN TestPrivateSharedIP I0924 12:50:36.205520 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:36.205616 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:36.205650 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:36.205697 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:36.205740 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:36.205787 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:36.205826 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:36.205855 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:36.205896 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:36.205923 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:36.222476 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:36.224328 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.224393 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.224448 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.296270 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:36.391016 65034 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0924 12:50:36.391640 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:36.391668 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:36.391767 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:36.394305 65034 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0924 12:50:36.394671 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:36.396320 65034 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0924 12:50:36.396519 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.396624 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.397032 65034 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0924 12:50:36.397102 65034 api.go:53] PageSize not implemented W0924 12:50:36.397269 65034 api.go:53] PageSize not implemented I0924 12:50:36.401845 65034 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0924 12:50:36.402023 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.402248 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.403476 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.404623 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.404737 65034 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0924 12:50:36.405022 65034 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0924 12:50:36.405212 65034 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0924 12:50:36.410845 65034 target.go:116] Terraform output is in /tmp/test1464584549/out --- PASS: TestPrivateSharedIP (0.21s) === RUN TestPrivateDns1 I0924 12:50:36.414871 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:36.414998 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:36.415037 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:36.415080 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:36.415127 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:36.415190 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:36.415273 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:36.415305 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:36.415343 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:36.415378 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:36.443287 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.443347 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.443382 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.597046 65034 executor.go:111] Tasks: 0 done / 120 total; 46 can run I0924 12:50:36.604092 65034 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0924 12:50:36.604439 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "internal.example.com" I0924 12:50:36.604469 65034 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0924 12:50:36.604526 65034 dnszone.go:248] Existing zone "internal.example.com." found; will configure TF to reuse I0924 12:50:36.604537 65034 dnszone.go:271] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0924 12:50:36.604797 65034 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0924 12:50:36.605042 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.605200 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.608374 65034 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0924 12:50:36.608433 65034 api.go:53] PageSize not implemented W0924 12:50:36.608655 65034 api.go:53] PageSize not implemented I0924 12:50:36.613425 65034 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0924 12:50:36.613578 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.613748 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.615014 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.616145 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.616261 65034 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0924 12:50:36.616567 65034 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0924 12:50:36.616754 65034 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0924 12:50:36.623016 65034 target.go:116] Terraform output is in /tmp/test3198806539/out --- PASS: TestPrivateDns1 (0.21s) === RUN TestPrivateDns2 I0924 12:50:36.626849 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:36.626979 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:36.627019 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:36.627062 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:36.627111 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:36.627165 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:36.627207 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:36.627238 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:36.627288 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:36.627318 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:36.658814 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.658893 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.658947 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.813085 65034 executor.go:111] Tasks: 0 done / 116 total; 44 can run I0924 12:50:36.816613 65034 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0924 12:50:36.816986 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "private.example.com" I0924 12:50:36.817016 65034 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0924 12:50:36.817072 65034 dnszone.go:248] Existing zone "private.example.com." found; will configure TF to reuse I0924 12:50:36.817084 65034 dnszone.go:262] VPC "vpc-12345678" already associated with zone "private.example.com." I0924 12:50:36.817007 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:36.817179 65034 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0924 12:50:36.817523 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.817742 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.819296 65034 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0924 12:50:36.819346 65034 api.go:53] PageSize not implemented W0924 12:50:36.819443 65034 api.go:53] PageSize not implemented I0924 12:50:36.821954 65034 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0924 12:50:36.822051 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.822238 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.822864 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.823553 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.823619 65034 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0924 12:50:36.823786 65034 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0924 12:50:36.823894 65034 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0924 12:50:36.830992 65034 target.go:116] Terraform output is in /tmp/test1096160612/out --- PASS: TestPrivateDns2 (0.21s) === RUN TestDiscoveryFeatureGate I0924 12:50:36.834778 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:36.834906 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:36.834945 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:36.835032 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:36.835075 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:36.835129 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:36.835171 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:36.835201 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:36.835240 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:36.835269 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:36.861467 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:36.864576 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.864790 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:36.936641 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:37.104543 65034 executor.go:111] Tasks: 0 done / 99 total; 51 can run I0924 12:50:37.105339 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:37.105382 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:37.105469 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:37.106820 65034 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0924 12:50:37.112279 65034 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0924 12:50:37.112508 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.112722 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.114235 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.114398 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.114509 65034 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0924 12:50:37.114800 65034 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0924 12:50:37.114961 65034 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0924 12:50:37.121116 65034 target.go:116] Terraform output is in /tmp/test1693689400/out --- PASS: TestDiscoveryFeatureGate (0.29s) === RUN TestVFSServiceAccountIssuerDiscovery I0924 12:50:37.126413 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:37.126545 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:37.126586 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:37.127264 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:37.127364 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:37.127454 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:37.127554 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:37.127647 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:37.127742 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:37.127814 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:37.150204 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:37.152902 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.152986 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.226300 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:37.307415 65034 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0924 12:50:37.308124 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:37.308148 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:37.308230 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:37.309508 65034 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0924 12:50:37.314280 65034 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0924 12:50:37.314546 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.314752 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.315731 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.316165 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.316236 65034 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0924 12:50:37.316416 65034 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0924 12:50:37.316569 65034 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0924 12:50:37.323736 65034 target.go:116] Terraform output is in /tmp/test2458642066/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.20s) === RUN TestAWSLBController I0924 12:50:37.327070 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:37.327198 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:37.327238 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:37.327290 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:37.327331 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:37.327383 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:37.327465 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:37.327498 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:37.327539 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:37.327568 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:37.352700 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:37.355594 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.355679 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:37.427458 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:37.998037 65034 executor.go:111] Tasks: 0 done / 96 total; 49 can run I0924 12:50:37.999834 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:38.000048 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:38.000230 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:38.005802 65034 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0924 12:50:38.010632 65034 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0924 12:50:38.010922 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.011106 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.011857 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.012631 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.012711 65034 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0924 12:50:38.012900 65034 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0924 12:50:38.013042 65034 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0924 12:50:38.016565 65034 target.go:116] Terraform output is in /tmp/test2488029162/out --- PASS: TestAWSLBController (0.70s) === RUN TestManyAddons I0924 12:50:38.023312 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:38.023467 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:38.023575 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:38.023712 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:38.023779 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:38.023860 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:38.023984 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:38.024027 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:38.024131 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:38.024197 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:38.042043 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:38.044444 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.044508 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.116196 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:38.730265 65034 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0924 12:50:38.730833 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:38.730866 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:38.730957 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:38.737407 65034 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0924 12:50:38.743004 65034 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0924 12:50:38.743335 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.743544 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.745202 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.745733 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.745895 65034 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0924 12:50:38.746243 65034 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0924 12:50:38.746549 65034 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0924 12:50:38.752925 65034 target.go:116] Terraform output is in /tmp/test3886457999/out --- PASS: TestManyAddons (0.74s) === RUN TestManyAddonsCCMIRSA I0924 12:50:38.761392 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:38.761523 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:38.761585 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:38.761644 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:38.761672 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:38.761703 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:38.761814 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:38.761847 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:38.761886 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:38.761905 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:38.787833 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:38.790253 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.790310 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:38.861813 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:39.596474 65034 executor.go:111] Tasks: 0 done / 123 total; 61 can run I0924 12:50:39.597058 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:39.597084 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:39.597158 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:39.598759 65034 executor.go:111] Tasks: 61 done / 123 total; 30 can run I0924 12:50:39.601679 65034 executor.go:111] Tasks: 91 done / 123 total; 26 can run I0924 12:50:39.601916 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:39.603124 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:39.604679 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:39.607187 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:39.607294 65034 executor.go:111] Tasks: 117 done / 123 total; 2 can run I0924 12:50:39.607509 65034 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0924 12:50:39.607715 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:50:39.612292 65034 target.go:116] Terraform output is in /tmp/test1179499987/out --- PASS: TestManyAddonsCCMIRSA (0.86s) === RUN TestManyAddonsCCMIRSA23 I0924 12:50:39.618889 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:39.618985 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:39.619019 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:39.619080 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:39.619108 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:39.619145 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:39.619179 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:39.619198 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:39.619223 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:39.619241 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:39.635609 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0924 12:50:39.637679 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:39.637740 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:39.718989 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:40.350946 65034 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0924 12:50:40.351409 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:40.351441 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:40.351526 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:40.353290 65034 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0924 12:50:40.359609 65034 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0924 12:50:40.359887 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:40.360075 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:40.361207 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:40.362476 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:40.362568 65034 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0924 12:50:40.362783 65034 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0924 12:50:40.362967 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:40.369447 65034 target.go:116] Terraform output is in /tmp/test3896939932/out --- PASS: TestManyAddonsCCMIRSA23 (0.76s) === RUN TestManyAddonsCCMIRSA24 I0924 12:50:40.377964 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:40.378080 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:40.378116 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:40.378154 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:40.378188 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:40.378231 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:40.378305 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:40.378331 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:40.378367 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:40.378391 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:40.401040 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:40.403108 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:40.403165 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:40.474044 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:41.073140 65034 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0924 12:50:41.073896 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:41.073923 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:41.074006 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:41.075516 65034 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0924 12:50:41.082210 65034 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0924 12:50:41.082472 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.082641 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.083385 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.084061 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.084127 65034 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0924 12:50:41.084295 65034 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0924 12:50:41.084532 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:41.089032 65034 target.go:116] Terraform output is in /tmp/test579652536/out --- PASS: TestManyAddonsCCMIRSA24 (0.72s) === RUN TestManyAddonsCCMIRSA25 I0924 12:50:41.095795 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:41.095883 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:41.095912 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:41.095943 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:41.095969 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:41.096003 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:41.096062 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:41.096080 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:41.096106 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:41.096124 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:41.112025 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:41.114078 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.114135 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.188225 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:41.795088 65034 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0924 12:50:41.795945 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:41.795987 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:41.796071 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:41.797245 65034 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0924 12:50:41.800501 65034 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0924 12:50:41.800718 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.800853 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.801580 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.803771 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.804011 65034 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0924 12:50:41.804623 65034 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0924 12:50:41.805232 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:41.810792 65034 target.go:116] Terraform output is in /tmp/test1329743081/out --- PASS: TestManyAddonsCCMIRSA25 (0.72s) === RUN TestManyAddonsCCMIRSA26 I0924 12:50:41.817242 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:41.817326 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:41.817355 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:41.817415 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:41.817442 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:41.817474 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:41.817501 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:41.817519 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:41.817541 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:41.817559 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:41.833592 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:41.835663 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.835719 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:41.915424 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:42.587315 65034 executor.go:111] Tasks: 0 done / 124 total; 62 can run I0924 12:50:42.588125 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:42.588157 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:42.588239 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:42.589506 65034 executor.go:111] Tasks: 62 done / 124 total; 30 can run I0924 12:50:42.592891 65034 executor.go:111] Tasks: 92 done / 124 total; 26 can run I0924 12:50:42.593274 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:42.593465 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:42.594520 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:42.595036 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:42.595117 65034 executor.go:111] Tasks: 118 done / 124 total; 2 can run I0924 12:50:42.595304 65034 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0924 12:50:42.595528 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:42.604364 65034 target.go:116] Terraform output is in /tmp/test2092643162/out --- PASS: TestManyAddonsCCMIRSA26 (0.79s) === RUN TestManyAddonsGCE I0924 12:50:42.611042 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:43.308703 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:43.308767 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:43.308838 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/main" W0924 12:50:43.308859 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com/backups/etcd/events" W0924 12:50:43.308873 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal.example.com" W0924 12:50:43.308920 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:43.309809 65034 executor.go:111] Tasks: 0 done / 67 total; 44 can run I0924 12:50:43.314792 65034 executor.go:111] Tasks: 44 done / 67 total; 17 can run I0924 12:50:43.318863 65034 executor.go:111] Tasks: 61 done / 67 total; 4 can run I0924 12:50:43.320477 65034 executor.go:111] Tasks: 65 done / 67 total; 2 can run I0924 12:50:43.320639 65034 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0924 12:50:43.323155 65034 target.go:116] Terraform output is in /tmp/test3230379737/out --- PASS: TestManyAddonsGCE (0.72s) === RUN TestCCM I0924 12:50:43.327520 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:43.327612 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:43.327638 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:43.327693 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:43.327720 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:43.327752 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:43.327776 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:43.327794 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:43.327819 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:43.327837 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:43.343313 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:43.345311 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:43.345363 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:43.416211 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:44.109230 65034 executor.go:111] Tasks: 0 done / 110 total; 53 can run I0924 12:50:44.112948 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:44.112978 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:44.113088 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:44.116765 65034 executor.go:111] Tasks: 53 done / 110 total; 24 can run I0924 12:50:44.119572 65034 executor.go:111] Tasks: 77 done / 110 total; 27 can run I0924 12:50:44.119816 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.119992 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.120708 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.122746 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.122882 65034 executor.go:111] Tasks: 104 done / 110 total; 2 can run I0924 12:50:44.123144 65034 executor.go:111] Tasks: 106 done / 110 total; 4 can run I0924 12:50:44.123462 65034 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0924 12:50:44.129937 65034 target.go:116] Terraform output is in /tmp/test1043333465/out --- PASS: TestCCM (0.81s) === RUN TestExternalDNS I0924 12:50:44.141168 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.141276 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:44.141305 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:44.141337 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:44.141364 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:44.141396 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:44.141428 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:44.141446 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:44.141469 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:44.141489 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:44.156669 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:44.158276 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.158333 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.230525 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:44.322273 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:44.324865 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:44.324895 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:44.324978 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:44.329282 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:44.334424 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:44.334891 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.335094 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.336576 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.337419 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.337535 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:44.337806 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:44.338057 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:44.342662 65034 target.go:116] Terraform output is in /tmp/test1933961170/out --- PASS: TestExternalDNS (0.20s) === RUN TestExternalDNSIRSA I0924 12:50:44.346157 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.346287 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:44.346326 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:44.346371 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:44.346417 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:44.346471 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:44.346514 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:44.346543 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:44.346581 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:44.346609 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:44.367613 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:44.369669 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.369728 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.441426 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:44.540803 65034 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0924 12:50:44.541596 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:44.541665 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:44.541826 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:44.543457 65034 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0924 12:50:44.551516 65034 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0924 12:50:44.551839 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.552066 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.553272 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.553924 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.554089 65034 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0924 12:50:44.554470 65034 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0924 12:50:44.554628 65034 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0924 12:50:44.558840 65034 target.go:116] Terraform output is in /tmp/test2245311892/out --- PASS: TestExternalDNSIRSA (0.22s) === RUN TestKarpenter I0924 12:50:44.562630 65034 featureflag.go:166] FeatureFlag "Karpenter"=true I0924 12:50:44.562674 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.562800 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:44.562842 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:44.562936 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:44.562979 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:44.563036 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:44.563081 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:44.563116 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:44.563157 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:44.563189 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:44.595414 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:44.599093 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.599183 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.599245 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.671774 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:44.837831 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.837951 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.838052 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.934370 65034 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0924 12:50:44.935022 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:44.935049 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:44.935139 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:44.936737 65034 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0924 12:50:44.941302 65034 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0924 12:50:44.941833 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.942280 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.942440 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.942951 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.945965 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.946137 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.946241 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.947461 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.947577 65034 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0924 12:50:44.947842 65034 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0924 12:50:44.947997 65034 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0924 12:50:44.952476 65034 target.go:116] Terraform output is in /tmp/test2093478421/out I0924 12:50:44.955830 65034 featureflag.go:166] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.39s) === RUN TestSharedSubnet I0924 12:50:44.955974 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.956078 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:44.956114 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:44.956153 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:44.956180 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:44.956215 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:44.956244 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:44.956263 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:44.956287 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:44.956305 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:44.972086 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:44.974815 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:44.974905 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.178066 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:45.247118 65034 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0924 12:50:45.247729 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:45.247759 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:45.247851 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:45.253077 65034 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0924 12:50:45.253487 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:45.256135 65034 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0924 12:50:45.256463 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.256628 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.257428 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.258076 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.258147 65034 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0924 12:50:45.258384 65034 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0924 12:50:45.258517 65034 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0924 12:50:45.262537 65034 target.go:116] Terraform output is in /tmp/test3349329138/out --- PASS: TestSharedSubnet (0.31s) === RUN TestSharedVPC I0924 12:50:45.265855 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:45.265991 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:45.266038 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:45.266090 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:45.266135 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:45.266189 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:45.266235 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:45.266267 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:45.266307 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:45.266381 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:45.291299 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:45.293845 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.293922 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.365511 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:45.457443 65034 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0924 12:50:45.463662 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:45.463710 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:45.463820 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:45.467014 65034 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0924 12:50:45.467335 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:45.471633 65034 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0924 12:50:45.471930 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.472069 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.473445 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.474414 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.474522 65034 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0924 12:50:45.474808 65034 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0924 12:50:45.474957 65034 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0924 12:50:45.479151 65034 target.go:116] Terraform output is in /tmp/test3289805552/out --- PASS: TestSharedVPC (0.22s) === RUN TestSharedVPCIPv6 I0924 12:50:45.482132 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:45.482262 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:45.482304 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:45.482348 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:45.482383 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:45.482427 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:45.482504 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:45.482534 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:45.482568 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:45.482594 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:45.484597 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:45.484653 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:45.500928 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:45.500992 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:45.501064 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:45.503359 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.503418 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.574802 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:45.717703 65034 executor.go:111] Tasks: 0 done / 121 total; 45 can run I0924 12:50:45.718201 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:45.718248 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:45.718331 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:45.720022 65034 executor.go:111] Tasks: 45 done / 121 total; 25 can run I0924 12:50:45.720397 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:45.720530 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:45.722024 65034 executor.go:111] Tasks: 70 done / 121 total; 33 can run I0924 12:50:45.722497 65034 executor.go:111] Tasks: 103 done / 121 total; 6 can run I0924 12:50:45.722528 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:45.722573 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:45.722702 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:45.722760 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:45.725378 65034 executor.go:111] Tasks: 109 done / 121 total; 8 can run I0924 12:50:45.725588 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.725754 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.726516 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.727339 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.727415 65034 executor.go:111] Tasks: 117 done / 121 total; 2 can run I0924 12:50:45.727632 65034 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0924 12:50:45.727755 65034 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0924 12:50:45.731346 65034 target.go:116] Terraform output is in /tmp/test2905193616/out --- PASS: TestSharedVPCIPv6 (0.25s) === RUN TestExistingIAM I0924 12:50:45.736790 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:45.736896 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:45.736927 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:45.736962 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:45.736990 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:45.737024 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:45.737054 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:45.737071 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:45.737096 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:45.737116 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:45.757309 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:45.760399 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.760481 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.760545 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.760601 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.832811 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:45.909144 65034 executor.go:111] Tasks: 0 done / 101 total; 51 can run I0924 12:50:45.919962 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:45.919994 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:45.920073 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:45.924726 65034 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0924 12:50:45.935767 65034 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0924 12:50:45.936201 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.938661 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.939258 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.941478 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.942115 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.942975 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.943373 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.945470 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.945631 65034 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0924 12:50:45.946305 65034 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0924 12:50:45.946592 65034 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0924 12:50:45.951909 65034 target.go:116] Terraform output is in /tmp/test2595664629/out --- PASS: TestExistingIAM (0.22s) === RUN TestPhaseNetwork I0924 12:50:45.955467 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:45.955617 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:45.955662 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:45.955709 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:45.955748 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:45.955794 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:45.955838 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:45.955868 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:45.955907 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:45.955935 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:45.979343 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:45.981731 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.981794 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:45.981831 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.053863 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:46.120511 65034 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0924 12:50:46.121658 65034 executor.go:111] Tasks: 46 done / 119 total; 26 can run I0924 12:50:46.123945 65034 executor.go:111] Tasks: 72 done / 119 total; 32 can run I0924 12:50:46.124759 65034 executor.go:111] Tasks: 104 done / 119 total; 5 can run I0924 12:50:46.124942 65034 executor.go:111] Tasks: 109 done / 119 total; 6 can run I0924 12:50:46.125138 65034 executor.go:111] Tasks: 115 done / 119 total; 2 can run I0924 12:50:46.125190 65034 executor.go:111] Tasks: 117 done / 119 total; 2 can run I0924 12:50:46.125228 65034 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0924 12:50:46.126103 65034 target.go:116] Terraform output is in /tmp/test3179176508/out --- PASS: TestPhaseNetwork (0.17s) === RUN TestExternalLoadBalancer I0924 12:50:46.126534 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:46.126637 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:46.126673 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:46.126716 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:46.126757 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:46.126807 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:46.126848 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:46.126878 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:46.126914 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:46.126941 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:46.149837 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:46.152262 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.152341 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.226046 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:46.312670 65034 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0924 12:50:46.313428 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:46.313456 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:46.313527 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:46.315008 65034 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0924 12:50:46.317851 65034 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0924 12:50:46.318133 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.318348 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.319824 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.320911 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.321022 65034 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0924 12:50:46.321313 65034 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0924 12:50:46.321458 65034 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0924 12:50:46.328054 65034 target.go:116] Terraform output is in /tmp/test615473541/out --- PASS: TestExternalLoadBalancer (0.20s) === RUN TestPhaseIAM integration_test.go:920: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:928: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0924 12:50:46.330150 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:46.330227 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:46.330253 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:46.330282 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:46.330308 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:46.330341 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:46.330366 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:46.330383 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:46.330407 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:46.330424 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:46.350375 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:46.352543 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.352599 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.352634 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.352667 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.352698 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.352724 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.352755 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.425116 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:46.526448 65034 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0924 12:50:46.526881 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:46.526911 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:46.527009 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:46.528741 65034 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0924 12:50:46.539073 65034 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0924 12:50:46.539724 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.539927 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.540151 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.540310 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.541269 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.541558 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.541708 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.542138 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.542243 65034 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0924 12:50:46.542523 65034 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0924 12:50:46.542685 65034 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0924 12:50:46.546530 65034 target.go:116] Terraform output is in /tmp/test679577449/out --- PASS: TestMixedInstancesASG (0.22s) === RUN TestMixedInstancesSpotASG I0924 12:50:46.549162 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:46.549243 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:46.549270 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:46.549303 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:46.549329 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:46.549361 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:46.549389 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:46.549408 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:46.549435 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:46.549454 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:46.568365 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:46.570544 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.570601 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.570636 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.570669 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.570700 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.570726 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.570751 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.642337 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:46.743401 65034 executor.go:111] Tasks: 0 done / 111 total; 53 can run I0924 12:50:46.744162 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:46.744209 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:46.744320 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:46.745749 65034 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0924 12:50:46.750634 65034 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0924 12:50:46.750833 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.750974 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.751101 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.751271 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.752212 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.752790 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.753171 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.753242 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.753311 65034 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0924 12:50:46.753608 65034 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0924 12:50:46.753811 65034 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0924 12:50:46.760984 65034 target.go:116] Terraform output is in /tmp/test1145278763/out --- PASS: TestMixedInstancesSpotASG (0.22s) === RUN TestAdditionalObjects I0924 12:50:46.766094 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:46.766234 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:46.766278 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:46.766331 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:46.766381 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:46.766440 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:46.766530 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:46.766566 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:46.766610 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:46.766644 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:46.795270 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.15 ********************************************************************************* I0924 12:50:46.798359 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.798448 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:46.918553 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:47.063263 65034 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0924 12:50:47.063886 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:47.063916 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:47.064013 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:47.065399 65034 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0924 12:50:47.068438 65034 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0924 12:50:47.068584 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.068709 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.070085 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.070216 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.070302 65034 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0924 12:50:47.070536 65034 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0924 12:50:47.070666 65034 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0924 12:50:47.073444 65034 target.go:116] Terraform output is in /tmp/test3747068149/out --- PASS: TestAdditionalObjects (0.31s) === RUN TestContainerd I0924 12:50:47.075804 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:47.075900 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:47.075932 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:47.075976 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:47.076013 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:47.076062 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:47.076100 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:47.076130 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:47.076170 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:47.076196 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:47.095041 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:47.096689 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.096749 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.170350 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:47.259630 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:47.260338 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:47.260373 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:47.260456 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:47.261479 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:47.264382 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:47.264614 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.264815 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.266179 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.266859 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.266974 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:47.267388 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:47.267549 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:47.280010 65034 target.go:116] Terraform output is in /tmp/test943568269/out --- PASS: TestContainerd (0.21s) === RUN TestContainerdCustom I0924 12:50:47.283609 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:47.283775 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:47.283853 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:47.283929 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:47.283999 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:47.284074 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:47.284148 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:47.284205 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:47.284274 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:47.284330 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:47.302812 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:47.304573 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.304650 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.375746 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:47.438008 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:47.438432 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:47.438477 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:47.438578 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:47.440844 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:47.445417 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:47.445653 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.445823 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.446607 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.447512 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.447588 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:47.447777 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:47.447945 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:47.450825 65034 target.go:116] Terraform output is in /tmp/test4186716645/out --- PASS: TestContainerdCustom (0.17s) === RUN TestDockerCustom I0924 12:50:47.454749 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:47.455152 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:47.455437 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:47.455507 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:47.455560 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:47.455623 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:47.455672 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:47.455708 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:47.455751 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:47.456248 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:47.483489 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:47.486610 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.486692 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.558326 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:47.640235 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:47.640565 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:47.640598 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:47.640692 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:47.642422 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:47.646666 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:47.646843 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.646987 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.648443 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.648585 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.648651 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:47.648879 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:47.648990 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:47.653733 65034 target.go:116] Terraform output is in /tmp/test670226106/out --- PASS: TestDockerCustom (0.20s) === RUN TestAPIServerNodes I0924 12:50:47.657144 65034 featureflag.go:166] FeatureFlag "APIServerNodes"=true I0924 12:50:47.657166 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:47.657252 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:47.657281 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:47.657315 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:47.657341 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:47.657373 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:47.657426 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:47.657444 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:47.657468 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:47.657486 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:47.673907 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.17 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.17 ********************************************************************************* I0924 12:50:47.676611 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.676667 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:47.676702 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.360117 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:48.457125 65034 executor.go:111] Tasks: 0 done / 100 total; 47 can run I0924 12:50:48.457552 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:48.457577 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:48.457671 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:48.459216 65034 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0924 12:50:48.463176 65034 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0924 12:50:48.463326 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.463449 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.463580 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.464261 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.464618 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.464998 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.465072 65034 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0924 12:50:48.465285 65034 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0924 12:50:48.465451 65034 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0924 12:50:48.468553 65034 target.go:116] Terraform output is in /tmp/test3149919422/out I0924 12:50:48.470838 65034 featureflag.go:166] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.81s) === RUN TestNTHIMDSProcessor I0924 12:50:48.470918 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:48.471001 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:48.471031 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:48.471064 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:48.471087 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:48.471131 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:48.471189 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:48.471216 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:48.471247 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:48.471284 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:48.487215 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:48.489029 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.489092 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.560868 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:48.660818 65034 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0924 12:50:48.661579 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:48.661606 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:48.661695 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:48.662851 65034 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0924 12:50:48.665989 65034 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0924 12:50:48.666166 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.666349 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.667276 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.667667 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.667742 65034 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0924 12:50:48.667980 65034 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0924 12:50:48.668099 65034 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0924 12:50:48.671228 65034 target.go:116] Terraform output is in /tmp/test3726298947/out --- PASS: TestNTHIMDSProcessor (0.20s) === RUN TestNTHIMDSProcessorIRSA I0924 12:50:48.673669 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:48.673747 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:48.673772 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:48.673802 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:48.673826 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:48.673859 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:48.673910 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:48.673927 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:48.673952 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:48.673969 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:48.692700 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:48.694615 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.694689 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.766125 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:48.883796 65034 executor.go:111] Tasks: 0 done / 95 total; 48 can run I0924 12:50:48.887392 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:48.887443 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:48.887583 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:48.889765 65034 executor.go:111] Tasks: 48 done / 95 total; 22 can run I0924 12:50:48.892731 65034 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0924 12:50:48.892979 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.893166 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.894515 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.895059 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.895135 65034 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0924 12:50:48.895348 65034 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0924 12:50:48.895469 65034 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0924 12:50:48.898512 65034 target.go:116] Terraform output is in /tmp/test30270351/out --- PASS: TestNTHIMDSProcessorIRSA (0.23s) === RUN TestCustomIRSA I0924 12:50:48.900993 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:48.901074 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:48.901100 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:48.901129 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:48.901153 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:48.901193 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:48.901257 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:48.901278 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:48.901306 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:48.901323 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:48.918200 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:48.920075 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.920128 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:48.992184 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:49.425760 65034 executor.go:111] Tasks: 0 done / 100 total; 50 can run I0924 12:50:49.426535 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:49.426572 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:49.428327 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:49.432406 65034 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0924 12:50:49.435561 65034 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0924 12:50:49.435913 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.436157 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.436905 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.437723 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.437805 65034 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0924 12:50:49.438120 65034 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0924 12:50:49.438304 65034 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0924 12:50:49.441791 65034 target.go:116] Terraform output is in /tmp/test2291378189/out --- PASS: TestCustomIRSA (0.55s) === RUN TestClusterNameDigit I0924 12:50:49.448926 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:49.449038 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:49.449072 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:49.449107 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:49.449133 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:49.449168 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:49.449198 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:49.449215 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:49.449238 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:49.449258 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:49.465815 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:49.467621 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.467680 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.539187 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:49.602859 65034 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0924 12:50:49.604639 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:49.604681 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:49.604786 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:49.609034 65034 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0924 12:50:49.611904 65034 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0924 12:50:49.612151 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.612296 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.613028 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.613697 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.613763 65034 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0924 12:50:49.613923 65034 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0924 12:50:49.614042 65034 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0924 12:50:49.616995 65034 target.go:116] Terraform output is in /tmp/test2285214064/out --- PASS: TestClusterNameDigit (0.17s) === RUN TestCASPriorityExpander I0924 12:50:49.619284 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:49.619362 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:49.619387 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:49.619415 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:49.619438 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:49.619471 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:49.619522 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:49.619540 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:49.619563 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:49.619579 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:49.637994 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:49.641677 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.641737 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.641771 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:49.641803 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0924 12:50:50.280188 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:50.441668 65034 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0924 12:50:50.442799 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:50.442831 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:50.442958 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:50.446222 65034 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0924 12:50:50.449306 65034 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0924 12:50:50.449470 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.449616 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.449688 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.449876 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.450460 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.450654 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.450780 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.451787 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.451885 65034 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0924 12:50:50.452153 65034 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0924 12:50:50.452304 65034 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0924 12:50:50.457164 65034 target.go:116] Terraform output is in /tmp/test761763036/out --- PASS: TestCASPriorityExpander (0.84s) === RUN TestCASPriorityExpanderCustom I0924 12:50:50.460814 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:50.460916 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:50.460951 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:50.460989 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:50.461024 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:50.461068 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:50.461146 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:50.461177 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:50.461213 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:50.461238 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:50.490051 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* I0924 12:50:50.492089 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.492150 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.492186 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.492219 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0924 12:50:50.568985 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:50.732358 65034 executor.go:111] Tasks: 0 done / 102 total; 48 can run I0924 12:50:50.733049 65034 dnszone.go:241] Check for existing route53 zone to re-use with name "" I0924 12:50:50.733076 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:50.733148 65034 dnszone.go:248] Existing zone "example.com." found; will configure TF to reuse I0924 12:50:50.734689 65034 executor.go:111] Tasks: 48 done / 102 total; 21 can run I0924 12:50:50.738516 65034 executor.go:111] Tasks: 69 done / 102 total; 25 can run I0924 12:50:50.738803 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.738985 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.739181 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.739341 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.740070 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.740114 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.740201 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.740549 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.740627 65034 executor.go:111] Tasks: 94 done / 102 total; 4 can run I0924 12:50:50.740871 65034 executor.go:111] Tasks: 98 done / 102 total; 4 can run I0924 12:50:50.741795 65034 executor.go:111] Tasks: 102 done / 102 total; 0 can run I0924 12:50:50.747944 65034 target.go:116] Terraform output is in /tmp/test2055652614/out --- PASS: TestCASPriorityExpanderCustom (0.29s) === RUN TestLifecycleMinimalAWS I0924 12:50:50.750583 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:50.750672 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:50.750703 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:50.750738 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:50.750765 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:50.750797 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:50.750825 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:50.750843 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:50.750867 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:50.750885 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:50.750917 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0924 12:50:50.764954 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:50.766613 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.766673 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0924 12:50:50.838342 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:50.906706 65034 iamrole.go:89] ListRoles: { } I0924 12:50:50.907589 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:50.908055 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:50.908068 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:50.908215 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0924 12:50:50.908279 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:50.908318 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0924 12:50:50.908421 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:50.908539 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:50.908535 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0924 12:50:50.908585 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:50.908628 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0924 12:50:50.908710 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.908793 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:50.908822 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:50.908928 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.909030 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:50.909119 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:50.909148 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0924 12:50:50.909251 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0924 12:50:50.909303 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:50:50.909325 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:50:50.909342 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:50.909365 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:50.909671 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:50.909770 65034 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0924 12:50:50.909793 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0924 12:50:50.909793 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] }] } I0924 12:50:50.909863 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:50:50.909905 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:50.909934 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:50:50.910012 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:50.910051 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:50.909988 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0924 12:50:50.910110 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:50:50.910188 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0924 12:50:50.910223 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:50.910232 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:50:50.910275 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:50.910324 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:50.910370 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0924 12:50:50.910454 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0924 12:50:50.910655 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I0924 12:50:50.910744 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:50.910790 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:50.910829 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:50.910962 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0924 12:50:50.911104 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I0924 12:50:50.911174 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:50.911265 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:50:50.911307 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:50.911347 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:50.911388 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0924 12:50:50.911443 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:50.911624 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:50:50.913131 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:50.913265 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.913365 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.913514 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.913586 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.913729 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.913870 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" }] }] } I0924 12:50:50.914912 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.915107 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.915219 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.915514 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.915737 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:50.915883 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:50.915992 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:50.916107 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.916302 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.916509 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.917655 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:50:50.917997 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:50:50.918054 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:50.918113 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:50.918161 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" }] }] } I0924 12:50:50.919014 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.919163 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" }] }] } I0924 12:50:50.919307 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.919432 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:50:50.919590 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:50.919830 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:50.919935 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:50.919970 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:50.919995 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:50.920037 65034 dns.go:232] Pre-creating DNS records I0924 12:50:50.920055 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:50.923245 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:50.924839 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:50.924894 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0924 12:50:50.995380 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:51.057898 65034 iamrole.go:89] ListRoles: { } I0924 12:50:51.058807 65034 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0924 12:50:51.059051 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.059166 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] }] } I0924 12:50:51.059288 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:51.059360 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:51.059398 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.059490 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.059549 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:51.059570 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:51.059601 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:51.060897 65034 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0924 12:50:51.061034 65034 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0924 12:50:51.061002 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:51.061097 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0924 12:50:51.061094 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0924 12:50:51.061266 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:51.061318 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.061423 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:51.061452 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0924 12:50:51.061543 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0924 12:50:51.061640 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:51.061675 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.061763 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:51.063641 65034 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0924 12:50:51.063800 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:51.063937 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:51.064112 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:51.064255 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:51.064344 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:51.064428 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:51.064516 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:51.066360 65034 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0924 12:50:51.066503 65034 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0924 12:50:51.066610 65034 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0924 12:50:51.068799 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:50:51.068848 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:51.068883 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:50:51.068914 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:50:51.068943 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:50:51.068975 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:50:51.069006 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:51.069038 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:50:51.069069 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:50:51.069104 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:50:51.069143 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:50:51.069174 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:50:51.069205 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:51.069240 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:50:51.069269 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:51.069301 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:51.069332 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:51.069367 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:50:51.069397 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:50:51.069430 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:50:51.069459 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:51.069494 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:51.069529 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:51.069562 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:50:51.070639 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:50:51.070702 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:50:51.070718 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:50:51.070752 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.070795 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0924 12:50:51.070842 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.070893 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.070933 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.070982 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.071019 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.071052 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0924 12:50:51.071099 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.071145 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:50:51.071182 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0924 12:50:51.071222 65034 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0924 12:50:51.071264 65034 api.go:53] PageSize not implemented I0924 12:50:51.071283 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:51.071307 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:51.071323 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:50:51.071347 65034 targetgroups.go:35] PageSize not implemented I0924 12:50:51.071362 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:50:51.071413 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:50:51.071444 65034 iamrole.go:89] ListRoles: { } I0924 12:50:51.071492 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:51.071516 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:51.071550 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:50:51.071586 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:51.071614 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:51.072029 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0924 12:50:51.072089 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:50:51.072066 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:50:51.072138 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0924 12:50:51.072132 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:51.072172 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:50:51.072204 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok iam-instance-profile:masters.minimal.example.com ok I0924 12:50:51.072119 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-group:nodes.minimal.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I0924 12:50:51.072179 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0924 12:50:51.072320 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:51.072340 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:50:51.072371 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:50:51.072421 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0924 12:50:51.072466 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:51.072508 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0924 12:50:51.072695 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:51.072715 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:50:51.072747 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0924 12:50:51.072834 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0924 12:50:51.072879 65034 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0924 12:50:51.072852 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:50:51.072896 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0924 12:50:51.072945 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0924 12:50:51.072955 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:50:51.072981 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0924 12:50:51.073004 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:50:51.073022 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0924 12:50:51.073057 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:50:51.073105 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:50:51.073144 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:50:51.073356 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.32s) === RUN TestLifecycleMinimalOpenstack I0924 12:50:51.073486 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0924 12:50:51.078609 65034 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0924 12:50:51.268435 65034 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0924 12:50:51.268483 65034 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0924 12:50:51.278206 65034 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0924 12:50:51.281179 65034 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0924 12:50:51.289009 65034 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0924 12:50:51.292107 65034 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/09/24 12:50:51 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:51 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:51 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0924 12:50:51.294411 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:51.294543 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:51.294820 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0924 12:50:52.300023 65034 executor.go:111] Tasks: 90 done / 90 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0924 12:50:52.482627 65034 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0924 12:50:52.482668 65034 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0924 12:50:52.494026 65034 executor.go:111] Tasks: 0 done / 90 total; 41 can run I0924 12:50:52.499513 65034 executor.go:111] Tasks: 41 done / 90 total; 41 can run I0924 12:50:52.505675 65034 executor.go:111] Tasks: 82 done / 90 total; 5 can run I0924 12:50:52.507016 65034 executor.go:111] Tasks: 87 done / 90 total; 3 can run 2023/09/24 12:50:52 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:52 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:52 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0924 12:50:52.509211 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:52.509799 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:52.510426 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0924 12:50:52.511354 65034 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0924 12:50:52.512703 65034 delete_cluster.go:128] Looking for cloud resources to delete 2023/09/24 12:50:52 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) Port:8f0d8154-ceb3-4271-8d5d-112d4348c85e ok ServerGroup:6a5be0b7-681e-4c58-967d-e1c7cdddbcf0 ok Network:98b4f86f-b6b4-4866-8fa3-98504d114877 ok Instance:0b8ddcaa-360e-49f6-92f4-f61f786bacd5 ok Port:efb7a037-5243-40d3-bf34-89243d1765b9 ok Port:e1b8a2b5-81e1-42fe-8a6f-fddfd7b0acb3 ok Subnet:713a727c-5711-4fa9-ae5a-7b6f764b551b ok Instance:e198df4d-ef24-42ab-9a77-ab2b5a35e79f ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:c2c4af42-1523-4dff-9c06-7e7975e4ed82 ok SecurityGroup:6e72ca33-662e-43c7-80ce-481a382ce35d ok SecurityGroup:e2c5fddc-3754-49d3-870c-22e08921c14c ok ServerGroup:de651363-f9a1-4911-82d0-7245fc9e950f ok Instance:284c83b5-84b0-44c7-b1e1-709b56aca868 ok --- PASS: TestLifecycleMinimalOpenstack (2.55s) === RUN TestLifecycleMinimalGCE I0924 12:50:53.620105 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0924 12:50:53.621293 65034 network.go:238] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:53.858540 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:53.858600 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:53.858685 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0924 12:50:53.858707 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0924 12:50:53.858721 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0924 12:50:53.858761 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:53.859620 65034 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0924 12:50:53.861702 65034 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0924 12:50:53.861756 65034 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0924 12:50:53.864802 65034 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0924 12:50:53.867336 65034 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0924 12:50:53.867544 65034 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0924 12:50:53.867635 65034 dns.go:232] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* W0924 12:50:54.138277 65034 external_access.go:39] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0924 12:50:54.138381 65034 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0924 12:50:54.138522 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0924 12:50:54.138554 65034 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0924 12:50:54.138578 65034 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0924 12:50:54.138655 65034 autoscalinggroup.go:130] enabling storage-rw for etcd backups I0924 12:50:54.140092 65034 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0924 12:50:54.142429 65034 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0924 12:50:54.147046 65034 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0924 12:50:54.148737 65034 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0924 12:50:54.148839 65034 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0924 12:50:54.150276 65034 delete_cluster.go:128] Looking for cloud resources to delete I0924 12:50:54.150321 65034 gce.go:99] Scanning zones: [us-test1-a] I0924 12:50:54.150687 65034 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0924 12:50:54.150706 65034 gce.go:1074] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:node-to-master-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok Disk:a-etcd-events-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok Disk:a-etcd-main-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1695559853 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1695559853 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.53s) === RUN TestLifecycleFloatingIPOpenstack I0924 12:50:54.151631 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0924 12:50:54.156617 65034 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0924 12:50:54.346053 65034 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0924 12:50:54.346113 65034 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0924 12:50:54.360548 65034 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0924 12:50:54.368886 65034 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0924 12:50:54.374241 65034 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0924 12:50:54.376019 65034 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/09/24 12:50:54 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:54 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:54 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0924 12:50:54.377724 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:54.377809 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:54.378196 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0924 12:50:55.382747 65034 executor.go:111] Tasks: 92 done / 92 total; 0 can run ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0924 12:50:55.599212 65034 pruning.go:114] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0924 12:50:55.599253 65034 pruning.go:114] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0924 12:50:55.613028 65034 executor.go:111] Tasks: 0 done / 92 total; 42 can run I0924 12:50:55.623406 65034 executor.go:111] Tasks: 42 done / 92 total; 41 can run I0924 12:50:55.635602 65034 executor.go:111] Tasks: 83 done / 92 total; 6 can run I0924 12:50:55.637223 65034 executor.go:111] Tasks: 89 done / 92 total; 3 can run 2023/09/24 12:50:55 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:55 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) 2023/09/24 12:50:55 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) W0924 12:50:55.639552 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:55.640174 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. W0924 12:50:55.640467 65034 bootstrapscript.go:164] exporting username and password. Consider using application credentials instead. I0924 12:50:55.641574 65034 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0924 12:50:55.642936 65034 delete_cluster.go:128] Looking for cloud resources to delete 2023/09/24 12:50:55 http: superfluous response.WriteHeader call from k8s.io/kops/cloudmock/openstack/mockcompute.(*MockClient).getServer (servers.go:167) Port:b6ee585f-c65f-4864-a642-8fb7a0e63e7e ok Instance:e646cd02-5d87-49e5-acda-93ac3d00cedd ok Port:0ddce50e-9bd5-4959-b656-a9691620e7de ok Instance:6efd6f42-3e7f-4d95-ad0f-937d95f01ff4 ok SecurityGroup:a5568e2a-7a44-4ec0-87dc-0b8d464d7c17 ok SecurityGroup:4a969261-bcaa-4b92-b019-56e38910ad4e ok Router-IF:8d620fe0-88e5-4341-b64d-397df35944f9 ok Router:faf2603a-9dc5-4c04-8e5f-88b93516cfd5 ok Port:ef57b5ba-0d51-4526-9fe3-78ad4f167ffe ok Subnet:8d620fe0-88e5-4341-b64d-397df35944f9 ok Instance:3ad3857d-d716-43b1-b58e-7bec449479bb ok Network:c1b4e3b0-99d2-4abf-9325-0fe71bac55c6 ok SecurityGroup:8fa1bcfe-c8fe-4b55-abda-318f086bb6c4 ok ServerGroup:f129dc15-e346-4dd2-b127-c8302fc83b07 ok Port:26b1093a-3381-4f91-96e3-20efb46ee060 ok ServerGroup:ad180396-18e9-41f9-b78f-c0163529a0af ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok --- PASS: TestLifecycleFloatingIPOpenstack (2.60s) === RUN TestLifecyclePrivateCalico I0924 12:50:56.753961 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:56.754068 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:56.754097 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:56.754138 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:56.754164 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:56.754198 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:56.754229 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:56.754249 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:56.754273 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:56.754291 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:56.754326 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0924 12:50:56.771616 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:56.773636 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:56.773694 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:56.773729 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:56.848901 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.133760 65034 iamrole.go:89] ListRoles: { } I0924 12:50:57.135088 65034 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0924 12:50:57.135443 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0924 12:50:57.135490 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:57.135479 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.135556 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0924 12:50:57.135625 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0924 12:50:57.135745 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0924 12:50:57.135816 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0924 12:50:57.135785 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:57.135873 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0924 12:50:57.135912 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0924 12:50:57.135931 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:57.135986 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.136126 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.136149 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:57.136294 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.137517 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:57.137700 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:57.137781 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.137896 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:50:57.138021 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.138195 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:57.138267 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:50:57.138326 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:57.138392 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.138748 65034 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0924 12:50:57.138960 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.139032 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0924 12:50:57.139082 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:50:57.139149 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0924 12:50:57.139161 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0924 12:50:57.139161 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0924 12:50:57.139200 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0924 12:50:57.139240 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0924 12:50:57.139316 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0924 12:50:57.139351 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:57.139390 65034 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0924 12:50:57.139395 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.139433 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0924 12:50:57.139655 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.139513 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatecalico-exa-hocohm", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatecalico-exa-hocohm" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0924 12:50:57.139900 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0924 12:50:57.140271 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0924 12:50:57.139794 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0924 12:50:57.140337 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0924 12:50:57.140433 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:50:57.140433 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0924 12:50:57.140508 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:57.140564 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I0924 12:50:57.140668 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.141195 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0924 12:50:57.141329 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0924 12:50:57.141442 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I0924 12:50:57.141525 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:57.141622 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.141723 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:57.141823 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.141826 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0924 12:50:57.141927 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:57.141979 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0924 12:50:57.141739 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0924 12:50:57.142048 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:57.142095 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0924 12:50:57.142150 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:57.142181 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:57.142125 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0924 12:50:57.142205 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:57.142256 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:50:57.142281 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:50:57.142302 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:57.142326 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:57.142346 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:57.142375 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:57.142405 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:57.142432 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:57.142456 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.142502 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:57.142532 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:57.142557 65034 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] } I0924 12:50:57.142630 65034 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0924 12:50:57.142759 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } W0924 12:50:57.142861 65034 api.go:53] PageSize not implemented I0924 12:50:57.142836 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:50:57.142863 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0924 12:50:57.142901 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:57.142977 65034 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.143011 65034 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatecalico-exa-hocohm", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }], Type: "network" } I0924 12:50:57.143007 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.143090 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0924 12:50:57.143158 65034 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0924 12:50:57.143278 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.143307 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0924 12:50:57.143333 65034 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0924 12:50:57.143403 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" }] }] } I0924 12:50:57.143428 65034 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0924 12:50:57.143615 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.143715 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.143814 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.144138 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0924 12:50:57.144318 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0924 12:50:57.144411 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.144473 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.144525 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.144621 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.144681 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:57.144725 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.144777 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1", Port: 22, Protocol: "TCP" } W0924 12:50:57.144847 65034 api.go:53] PageSize not implemented I0924 12:50:57.144861 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.144821 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0924 12:50:57.144945 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0924 12:50:57.145033 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.145129 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:50:57.145198 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatecalico.example.com" }] }] } I0924 12:50:57.145292 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:50:57.145329 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:50:57.145368 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.145463 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.145565 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.145654 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0924 12:50:57.145690 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0924 12:50:57.145725 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:57.145768 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.145859 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.145955 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.146047 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.146138 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0924 12:50:57.146212 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0924 12:50:57.146303 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0924 12:50:57.146390 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0924 12:50:57.146479 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:57.146521 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.146622 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" }] }] } I0924 12:50:57.146713 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:50:57.146780 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0924 12:50:57.146871 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" }] }] } I0924 12:50:57.146977 65034 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0924 12:50:57.147015 65034 api.go:53] PageSize not implemented I0924 12:50:57.147061 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.147057 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:57.147116 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.147175 65034 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } W0924 12:50:57.147195 65034 api.go:53] PageSize not implemented I0924 12:50:57.147161 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:57.147240 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.147260 65034 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0924 12:50:57.147329 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0924 12:50:57.147355 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:57.147378 65034 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0924 12:50:57.149939 65034 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0924 12:50:57.150046 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.150138 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0924 12:50:57.150177 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.150254 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.150417 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.150493 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.151729 65034 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0924 12:50:57.151888 65034 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0924 12:50:57.151982 65034 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0924 12:50:57.152012 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.152034 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:57.152077 65034 dns.go:232] Pre-creating DNS records I0924 12:50:57.152092 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:57.155662 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:57.157587 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.157636 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.157674 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.228396 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.542006 65034 iamrole.go:89] ListRoles: { } I0924 12:50:57.544121 65034 executor.go:111] Tasks: 0 done / 128 total; 50 can run I0924 12:50:57.544440 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.544623 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:57.544618 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0924 12:50:57.544795 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:57.544903 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.544989 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:57.545009 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:57.545039 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.545076 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.546293 65034 executor.go:111] Tasks: 50 done / 128 total; 26 can run I0924 12:50:57.546402 65034 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0924 12:50:57.546439 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0924 12:50:57.546467 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0924 12:50:57.546448 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] }] } I0924 12:50:57.546499 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatecalico-exa-hocohm"] } I0924 12:50:57.546575 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0924 12:50:57.546530 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0924 12:50:57.546615 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } I0924 12:50:57.546676 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:57.546720 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.546807 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:57.546831 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0924 12:50:57.546911 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:57.546943 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0924 12:50:57.547029 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:57.547066 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:57.547092 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.547151 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0924 12:50:57.547230 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:50:57.547269 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.547346 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.547422 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.547467 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I0924 12:50:57.547599 65034 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0924 12:50:57.547696 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0924 12:50:57.547779 65034 api.go:53] PageSize not implemented I0924 12:50:57.547706 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:57.547807 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:57.547799 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.547827 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0924 12:50:57.547857 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.547815 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.547859 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0924 12:50:57.547882 65034 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0924 12:50:57.547891 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.547908 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } I0924 12:50:57.547981 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4"] } I0924 12:50:57.548008 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } W0924 12:50:57.548060 65034 api.go:53] PageSize not implemented I0924 12:50:57.548071 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.548093 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.548148 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:50:57.548176 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0924 12:50:57.548202 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0924 12:50:57.548382 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.548520 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.548572 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.548651 65034 executor.go:111] Tasks: 113 done / 128 total; 5 can run W0924 12:50:57.548691 65034 api.go:53] PageSize not implemented I0924 12:50:57.548717 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.548716 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:57.548773 65034 dnsname.go:118] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0924 12:50:57.548790 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.548767 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.548821 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0924 12:50:57.548840 65034 api.go:53] PageSize not implemented I0924 12:50:57.548874 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.551623 65034 executor.go:111] Tasks: 118 done / 128 total; 6 can run I0924 12:50:57.551687 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.551780 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.551839 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.551938 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.552017 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.553477 65034 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0924 12:50:57.553531 65034 api.go:53] PageSize not implemented I0924 12:50:57.553546 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.553658 65034 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0924 12:50:57.553748 65034 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0924 12:50:57.555714 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:50:57.555752 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:57.555776 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:50:57.555796 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0924 12:50:57.555818 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:50:57.555838 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:50:57.555856 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:50:57.555878 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:57.555901 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:50:57.555921 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:50:57.555944 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:50:57.555967 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0924 12:50:57.555991 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0924 12:50:57.556013 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:50:57.556033 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:50:57.556052 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:57.556073 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:50:57.556094 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:57.556115 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:57.556137 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:57.556159 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:57.556181 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:50:57.556203 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:57.556225 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:50:57.556246 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:57.556269 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:50:57.556288 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:50:57.556307 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:57.556329 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:57.556350 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:57.556370 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:50:57.557247 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:50:57.557287 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:50:57.557299 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:50:57.557323 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557359 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0924 12:50:57.557400 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557437 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557466 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557496 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557521 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557546 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0924 12:50:57.557583 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557614 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:57.557632 65034 natgateway.go:117] DescribeNatGateways: { } I0924 12:50:57.557654 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0924 12:50:57.557678 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0924 12:50:57.557709 65034 api.go:53] PageSize not implemented I0924 12:50:57.557719 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0924 12:50:57.557741 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:57.557756 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:57.557765 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1"] } I0924 12:50:57.557789 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:50:57.557803 65034 targetgroups.go:35] PageSize not implemented I0924 12:50:57.557813 65034 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0924 12:50:57.557849 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:50:57.557872 65034 iamrole.go:89] ListRoles: { } I0924 12:50:57.557904 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.557933 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:57.557959 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:50:57.557988 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:57.558012 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0924 12:50:57.558034 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:50:57.558054 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-config:lt-2 ok I0924 12:50:57.558406 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } autoscaling-group:bastion.privatecalico.example.com ok I0924 12:50:57.558447 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0924 12:50:57.558463 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:50:57.558478 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0924 12:50:57.558513 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0924 12:50:57.558528 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0924 12:50:57.558536 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.privatecalico.example.com ok I0924 12:50:57.558554 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0924 12:50:57.558475 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatecalico-exa-hocohm/4 ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0924 12:50:57.558616 65034 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0924 12:50:57.558558 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0924 12:50:57.558667 65034 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatecalico-exa-hocohm/1 ok I0924 12:50:57.558678 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0924 12:50:57.558594 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0924 12:50:57.558718 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.558734 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:50:57.558757 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0924 12:50:57.558788 65034 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0924 12:50:57.558825 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:50:57.558855 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok I0924 12:50:57.558903 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:57.558931 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0924 12:50:57.559058 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.559069 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:50:57.559087 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:57.559112 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0924 12:50:57.559178 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.559185 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0924 12:50:57.559231 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0924 12:50:57.559261 65034 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0924 12:50:57.559254 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0924 12:50:57.559271 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } subnet:subnet-1 ok I0924 12:50:57.559294 65034 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0924 12:50:57.559297 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0924 12:50:57.559329 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:50:57.559353 65034 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0924 12:50:57.559365 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0924 12:50:57.559389 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0924 12:50:57.559415 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0924 12:50:57.559444 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0924 12:50:57.559358 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 } ] } I0924 12:50:57.559452 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } W0924 12:50:57.559466 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.559469 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0924 12:50:57.559486 65034 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0924 12:50:57.559478 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0924 12:50:57.559516 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0924 12:50:57.559575 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0924 12:50:57.559612 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:50:57.559654 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:50:57.559696 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:50:57.559855 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.81s) === RUN TestLifecyclePrivateKopeio I0924 12:50:57.559970 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:57.560052 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:57.560074 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:57.560099 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:57.560123 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:57.560153 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:57.560179 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:57.560197 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:57.560219 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:57.560236 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:57.560266 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0924 12:50:57.580501 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:57.582344 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.582401 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.582436 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.654980 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.736355 65034 iamrole.go:89] ListRoles: { } I0924 12:50:57.738026 65034 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0924 12:50:57.738177 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.738304 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0924 12:50:57.738392 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0924 12:50:57.738373 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.738492 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:57.738527 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0924 12:50:57.738545 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] }] } I0924 12:50:57.738599 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0924 12:50:57.738605 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:57.738691 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:57.738760 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0924 12:50:57.738843 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0924 12:50:57.738870 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:57.738985 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:57.739022 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.739123 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:57.739196 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:57.739224 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.739321 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0924 12:50:57.739365 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0924 12:50:57.739504 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:50:57.739530 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:50:57.739548 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:57.739575 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.740153 65034 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0924 12:50:57.740242 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.740347 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:50:57.740389 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0924 12:50:57.740362 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0924 12:50:57.740404 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0924 12:50:57.740425 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0924 12:50:57.740494 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.740463 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0924 12:50:57.740552 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.740523 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-privatekopeio-exa-d8ef8e", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-privatekopeio-exa-d8ef8e" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0924 12:50:57.740622 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0924 12:50:57.740609 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0924 12:50:57.740669 65034 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0924 12:50:57.740636 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0924 12:50:57.740690 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0924 12:50:57.740689 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] }] } I0924 12:50:57.740760 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0924 12:50:57.740798 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0924 12:50:57.740860 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0924 12:50:57.740832 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I0924 12:50:57.740831 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0924 12:50:57.740897 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0924 12:50:57.741009 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:57.741097 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:57.741141 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:57.741180 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.741355 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] }] } I0924 12:50:57.741360 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0924 12:50:57.741437 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0924 12:50:57.741475 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.741515 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0924 12:50:57.741571 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0924 12:50:57.741602 65034 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0924 12:50:57.741609 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0924 12:50:57.741661 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:57.741693 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:57.741716 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0924 12:50:57.741791 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:50:57.741817 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:57.741838 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0924 12:50:57.741955 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0924 12:50:57.742014 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.741817 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0924 12:50:57.742075 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.742133 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.742133 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0924 12:50:57.742182 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:57.742237 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:50:57.742272 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0924 12:50:57.742321 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:50:57.742350 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:50:57.742374 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:57.742407 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:57.742436 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:57.742463 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0924 12:50:57.742486 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.742539 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:57.742568 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:57.742598 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:57.742628 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:57.742651 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:57.742679 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] } I0924 12:50:57.742720 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:57.742751 65034 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0924 12:50:57.742795 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:57.742822 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:57.742867 65034 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0924 12:50:57.742933 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W0924 12:50:57.742989 65034 api.go:53] PageSize not implemented I0924 12:50:57.742989 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0924 12:50:57.743048 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:50:57.743063 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:57.743113 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:57.743107 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.743086 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0924 12:50:57.743184 65034 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" }] } I0924 12:50:57.743182 65034 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-privatekopeio-exa-d8ef8e", SubnetMappings: [{ SubnetId: "subnet-4" },{ SubnetId: "subnet-3" }], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }], Type: "network" } I0924 12:50:57.743244 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.743193 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.743281 65034 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0924 12:50:57.743328 65034 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0924 12:50:57.743362 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0924 12:50:57.743390 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-4" } I0924 12:50:57.743418 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.743490 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0924 12:50:57.743515 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0924 12:50:57.743545 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:50:57.743614 65034 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0924 12:50:57.743668 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:57.743721 65034 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0924 12:50:57.743764 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:57.743813 65034 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0924 12:50:57.743940 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.744106 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0924 12:50:57.744268 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:57.744347 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.744462 65034 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0924 12:50:57.744501 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:50:57.744531 65034 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0924 12:50:57.744555 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0924 12:50:57.744604 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-172.20.8.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0924 12:50:57.744687 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.744780 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.744824 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.744875 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.744912 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0924 12:50:57.744935 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745021 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745096 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745177 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745256 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745330 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0924 12:50:57.745390 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745475 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745601 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0924 12:50:57.745678 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0924 12:50:57.745707 65034 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0924 12:50:57.745735 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" }] }] } I0924 12:50:57.745815 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.745871 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.745943 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }] } I0924 12:50:57.745995 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.746076 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.746148 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.746225 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:50:57.746275 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:50:57.746297 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0924 12:50:57.746372 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:50:57.746427 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:57.746451 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0924 12:50:57.746476 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1", Port: 22, Protocol: "TCP" } W0924 12:50:57.746542 65034 api.go:53] PageSize not implemented I0924 12:50:57.746559 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.746530 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:57.746594 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:57.746623 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0924 12:50:57.746690 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0924 12:50:57.746718 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.746792 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0924 12:50:57.746837 65034 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0924 12:50:57.746883 65034 api.go:53] PageSize not implemented I0924 12:50:57.746915 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.746900 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0924 12:50:57.746983 65034 api.go:53] PageSize not implemented I0924 12:50:57.746961 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:57.747004 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-3" } I0924 12:50:57.747014 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.747039 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.747060 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:57.747153 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0924 12:50:57.752428 65034 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0924 12:50:57.752674 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.752760 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.752878 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.752978 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.754438 65034 executor.go:111] Tasks: 124 done / 128 total; 2 can run I0924 12:50:57.754669 65034 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0924 12:50:57.754770 65034 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0924 12:50:57.754807 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.754829 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:57.754872 65034 dns.go:232] Pre-creating DNS records I0924 12:50:57.754888 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:57.758497 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:57.760257 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.760307 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.760343 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.831930 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.926537 65034 iamrole.go:89] ListRoles: { } I0924 12:50:57.928259 65034 executor.go:111] Tasks: 0 done / 128 total; 47 can run I0924 12:50:57.928494 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:57.928652 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.928883 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.928884 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:57.928993 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.929057 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:57.929075 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:57.929105 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.929146 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.930572 65034 executor.go:111] Tasks: 47 done / 128 total; 29 can run I0924 12:50:57.930721 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0924 12:50:57.930705 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.930803 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0924 12:50:57.930937 65034 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0924 12:50:57.930955 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:57.930957 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-privatekopeio-exa-d8ef8e"] } I0924 12:50:57.931041 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0924 12:50:57.931046 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0924 12:50:57.931075 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } I0924 12:50:57.931019 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0924 12:50:57.931210 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:57.931281 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0924 12:50:57.931428 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:57.931488 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.931636 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:57.931695 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.931889 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:57.931997 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.932154 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:57.932201 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0924 12:50:57.932369 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:50:57.932442 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.932572 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:57.932633 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.932827 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.932966 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.933107 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0924 12:50:57.933154 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.933326 65034 executor.go:111] Tasks: 76 done / 128 total; 37 can run I0924 12:50:57.933482 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0924 12:50:57.933577 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:57.933642 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:57.933613 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0924 12:50:57.933657 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0924 12:50:57.933691 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0924 12:50:57.933719 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } W0924 12:50:57.933653 65034 api.go:53] PageSize not implemented I0924 12:50:57.933752 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4"] } I0924 12:50:57.933781 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0924 12:50:57.933690 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.933774 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.933864 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.933902 65034 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0924 12:50:57.933938 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0924 12:50:57.933984 65034 api.go:53] PageSize not implemented I0924 12:50:57.934000 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.934031 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.934147 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.934283 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:57.934356 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:57.934437 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.934501 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.934699 65034 executor.go:111] Tasks: 113 done / 128 total; 6 can run W0924 12:50:57.934787 65034 api.go:53] PageSize not implemented I0924 12:50:57.934778 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:57.934787 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:57.934915 65034 dnsname.go:118] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0924 12:50:57.934892 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:57.934878 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0924 12:50:57.935082 65034 api.go:53] PageSize not implemented I0924 12:50:57.935133 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.935294 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.939207 65034 executor.go:111] Tasks: 119 done / 128 total; 5 can run I0924 12:50:57.939298 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.939441 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.939500 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.939641 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:57.941925 65034 executor.go:111] Tasks: 124 done / 128 total; 2 can run W0924 12:50:57.942064 65034 api.go:53] PageSize not implemented I0924 12:50:57.942092 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.942278 65034 executor.go:111] Tasks: 126 done / 128 total; 2 can run I0924 12:50:57.942441 65034 executor.go:111] Tasks: 128 done / 128 total; 0 can run I0924 12:50:57.948343 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:50:57.948991 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:57.949040 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:50:57.949077 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:50:57.949111 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:50:57.949144 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:50:57.949179 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:57.949217 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:50:57.949252 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:50:57.949290 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:50:57.949329 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0924 12:50:57.949374 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:50:57.949406 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:50:57.949440 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:57.949475 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:50:57.949506 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:57.949543 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0924 12:50:57.949577 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:57.949611 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:57.949646 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:57.949684 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:50:57.949717 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:57.949753 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:50:57.949783 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:57.949821 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:50:57.949857 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:57.949894 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:50:57.949924 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:50:57.949956 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:57.949993 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:57.950031 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:57.950085 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:50:57.951089 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:50:57.951147 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:50:57.951167 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:50:57.951207 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951269 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0924 12:50:57.951327 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951381 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951422 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951468 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951505 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951552 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0924 12:50:57.951611 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951666 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:57.951697 65034 natgateway.go:117] DescribeNatGateways: { } I0924 12:50:57.951734 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0924 12:50:57.951770 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0924 12:50:57.951819 65034 api.go:53] PageSize not implemented I0924 12:50:57.951837 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0924 12:50:57.951885 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:57.951908 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:57.951924 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1"] } I0924 12:50:57.951958 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:50:57.951981 65034 targetgroups.go:35] PageSize not implemented I0924 12:50:57.951997 65034 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0924 12:50:57.952049 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:50:57.952088 65034 iamrole.go:89] ListRoles: { } I0924 12:50:57.952130 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:57.952155 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:57.952194 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:50:57.952233 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:57.952282 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-3","rtb-1"] } I0924 12:50:57.952326 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0924 12:50:57.952365 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-config:lt-1 ok I0924 12:50:57.952943 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0924 12:50:57.953015 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0924 12:50:57.953074 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0924 12:50:57.952984 65034 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1" } I0924 12:50:57.953127 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-privatekopeio-exa-d8ef8e/1 ok I0924 12:50:57.953107 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0924 12:50:57.953176 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4" } autoscaling-group:bastion.privatekopeio.example.com ok I0924 12:50:57.953177 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-privatekopeio-exa-d8ef8e/4 ok volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0924 12:50:57.953191 65034 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } iam-instance-profile:bastions.privatekopeio.example.com ok I0924 12:50:57.953296 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0924 12:50:57.953268 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } load-balancer:api-privatekopeio-example-tl2bv8 ok I0924 12:50:57.953178 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I0924 12:50:57.953382 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.8.0/22" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 } ] } W0924 12:50:57.953632 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.953662 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:50:57.953702 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:57.953760 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0924 12:50:57.953960 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.953981 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:50:57.954016 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0924 12:50:57.954069 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:57.954121 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0924 12:50:57.954243 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.954266 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-2 ok I0924 12:50:57.954330 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0924 12:50:57.954418 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:50:57.954494 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0924 12:50:57.954633 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0924 12:50:57.954653 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0924 12:50:57.954740 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0924 12:50:57.954814 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0924 12:50:57.954790 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:50:57.954834 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0924 12:50:57.954870 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0924 12:50:57.954917 65034 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0924 12:50:57.954954 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0924 12:50:57.954869 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:50:57.955004 65034 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok W0924 12:50:57.955025 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:57.955044 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0924 12:50:57.955051 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0924 12:50:57.955106 65034 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0924 12:50:57.955085 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0924 12:50:57.955121 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0924 12:50:57.955138 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:50:57.955153 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0924 12:50:57.955188 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0924 12:50:57.955276 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:50:57.955362 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0924 12:50:57.955477 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0924 12:50:57.955606 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:50:57.955664 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:50:57.955888 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.40s) === RUN TestLifecycleIPv6 I0924 12:50:57.956038 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:57.956147 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:57.956175 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:57.956206 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:57.956235 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:57.956271 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:57.956312 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:57.956348 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:57.956379 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:57.956400 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:57.956436 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0924 12:50:57.998298 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:58.002159 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.002251 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.073258 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.192790 65034 iamrole.go:89] ListRoles: { } I0924 12:50:58.194006 65034 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0924 12:50:58.194311 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0924 12:50:58.194419 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] } I0924 12:50:58.194560 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.194556 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:58.194646 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0924 12:50:58.194330 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.194724 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.194901 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.194922 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.195069 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:58.195169 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:58.195203 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:58.195304 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.195404 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:58.195480 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:58.195510 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.195609 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.195670 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:58.195705 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:50:58.195726 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:50:58.195745 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:58.195771 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:58.196193 65034 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0924 12:50:58.196299 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0924 12:50:58.196383 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0924 12:50:58.196309 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0924 12:50:58.196452 65034 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0924 12:50:58.196407 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0924 12:50:58.196431 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0924 12:50:58.196505 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0924 12:50:58.196515 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.196524 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:58.196583 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0924 12:50:58.196626 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:58.196673 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:58.196706 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.196726 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:58.196871 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:58.196935 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:58.196978 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.197086 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0924 12:50:58.197168 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0924 12:50:58.197204 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0924 12:50:58.197298 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:58.197342 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.minimal-ipv6.example.com" } ] } I0924 12:50:58.197426 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.197515 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:50:58.197595 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.197625 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0924 12:50:58.197705 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:58.197752 65034 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0924 12:50:58.197666 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0924 12:50:58.197839 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0924 12:50:58.197971 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0924 12:50:58.197987 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.198123 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:58.198181 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:58.198230 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:58.198274 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:50:58.198330 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:58.198370 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:58.198406 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:58.198434 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0924 12:50:58.198459 65034 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0924 12:50:58.198505 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.198608 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.198661 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:58.198689 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:50:58.198735 65034 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0924 12:50:58.198821 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.198902 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.198957 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.199151 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", Ipv6CidrBlock: "2001:db8:0:111::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0924 12:50:58.199174 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.199368 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.199598 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.199742 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0924 12:50:58.199884 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0924 12:50:58.200022 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.200182 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", Ipv6CidrBlock: "2001:db8:0:112::/64", Ipv6Native: true, TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0924 12:50:58.200270 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0924 12:50:58.200366 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:58.200402 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:58.200439 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0924 12:50:58.200658 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:50:58.200756 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.200823 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.200937 65034 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.200892 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0924 12:50:58.201036 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0924 12:50:58.201132 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.201281 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.201358 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.201427 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0924 12:50:58.201459 65034 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0924 12:50:58.201479 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.201576 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.201669 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.201770 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", Ipv6CidrBlock: "2001:db8:0:114::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "dualstack-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" } ] }], VpcId: "vpc-1" } I0924 12:50:58.201761 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.201864 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-3" } I0924 12:50:58.201680 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.4.0/22", Ipv6CidrBlock: "2001:db8:0:116::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0924 12:50:58.201910 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", EgressOnlyInternetGatewayId: "eigw-2", RouteTableId: "rtb-1" } I0924 12:50:58.201862 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:113::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "DualStack" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "Name", Value: "dualstack-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0924 12:50:58.201948 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0924 12:50:58.201963 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.0.0/22", Ipv6CidrBlock: "2001:db8:0:115::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0924 12:50:58.202061 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0924 12:50:58.202096 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0924 12:50:58.202121 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0924 12:50:58.202155 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0924 12:50:58.202260 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.202340 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.202432 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:50:58.202462 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:58.202486 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.202564 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.202651 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:58.202680 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:58.202716 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:58.202742 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0924 12:50:58.202774 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0924 12:50:58.202800 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0924 12:50:58.202823 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0924 12:50:58.202902 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.202975 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0924 12:50:58.203079 65034 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0924 12:50:58.203129 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.203175 65034 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:58.203214 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:50:58.203213 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.203265 65034 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-2", SubnetId: "subnet-4", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1b.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } W0924 12:50:58.203262 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.203323 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0924 12:50:58.203427 65034 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-6" },{ SubnetId: "subnet-4" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0924 12:50:58.203488 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:58.203531 65034 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0924 12:50:58.203562 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.203622 65034 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-6"] }] } I0924 12:50:58.203689 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0924 12:50:58.203692 65034 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-6", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0924 12:50:58.203719 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0924 12:50:58.203762 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0924 12:50:58.203818 65034 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0924 12:50:58.203845 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-6"] }] } W0924 12:50:58.203862 65034 api.go:53] PageSize not implemented I0924 12:50:58.203878 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-6" } I0924 12:50:58.203904 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.203990 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0924 12:50:58.204018 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-3" } I0924 12:50:58.204058 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.204129 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-5"] }] } I0924 12:50:58.204151 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-5" } I0924 12:50:58.204173 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.204242 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:50:58.204264 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:50:58.204289 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.204394 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0924 12:50:58.204424 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.204475 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0924 12:50:58.204506 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0924 12:50:58.204533 65034 tags.go:42] CreateTags { Resources: ["subnet-6"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-2" }] } I0924 12:50:58.204561 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0924 12:50:58.204602 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0924 12:50:58.204636 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I0924 12:50:58.204680 65034 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0924 12:50:58.204720 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.204720 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:58.204757 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.204771 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.204794 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.204768 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:58.204889 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-2", RouteTableId: "rtb-1" } I0924 12:50:58.204812 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:58.204928 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.204922 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-3" } I0924 12:50:58.204950 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.204943 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.204951 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:58.205020 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-2", RouteTableId: "rtb-1" } I0924 12:50:58.205047 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.205024 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:58.205110 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-3" } I0924 12:50:58.205153 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:58.207691 65034 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0924 12:50:58.207881 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.208015 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.208112 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.208189 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.209777 65034 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0924 12:50:58.209999 65034 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0924 12:50:58.210105 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:58.210145 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.210178 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:58.210238 65034 dns.go:232] Pre-creating DNS records I0924 12:50:58.210261 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:58.217498 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.25.5 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.25.5 ********************************************************************************* I0924 12:50:58.222139 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.222226 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.296620 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.424079 65034 iamrole.go:89] ListRoles: { } I0924 12:50:58.425422 65034 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0924 12:50:58.425713 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.426976 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.427262 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:58.427260 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.427446 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:58.427485 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:58.427631 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:58.427749 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.429567 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:58.431350 65034 executor.go:111] Tasks: 46 done / 124 total; 21 can run I0924 12:50:58.431502 65034 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0924 12:50:58.431495 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0924 12:50:58.431464 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0924 12:50:58.431565 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0924 12:50:58.431603 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0924 12:50:58.431561 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0924 12:50:58.431588 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0924 12:50:58.431711 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:58.431749 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:58.431805 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.431886 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0924 12:50:58.431974 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:58.432023 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.432120 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.432199 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:58.432258 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.432335 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:58.432371 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.432737 65034 executor.go:111] Tasks: 67 done / 124 total; 32 can run I0924 12:50:58.432875 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.432963 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.433031 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:50:58.433059 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0924 12:50:58.433085 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0924 12:50:58.433136 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.433209 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.433292 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:58.433337 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.433401 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.433483 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0924 12:50:58.433508 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.433558 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.433639 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:58.433663 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.433704 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0924 12:50:58.433724 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0924 12:50:58.433745 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0924 12:50:58.433862 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.433974 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-6"] } I0924 12:50:58.434001 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.434108 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["dualstack-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.434192 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-5"] } I0924 12:50:58.434215 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:58.434299 65034 executor.go:111] Tasks: 99 done / 124 total; 9 can run I0924 12:50:58.434354 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.434382 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:58.434423 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.434433 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.434431 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.434457 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.434481 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0924 12:50:58.434480 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.434504 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0924 12:50:58.434525 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0924 12:50:58.434526 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.434573 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } W0924 12:50:58.434579 65034 api.go:53] PageSize not implemented I0924 12:50:58.434639 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.434691 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.434730 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:50:58.434763 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.434806 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:58.434868 65034 executor.go:111] Tasks: 108 done / 124 total; 8 can run I0924 12:50:58.434903 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:50:58.434911 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0924 12:50:58.434938 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.434927 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:58.435000 65034 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0924 12:50:58.434971 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.434982 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.435021 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:58.435037 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:58.435069 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.435070 65034 dnsname.go:118] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0924 12:50:58.435051 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0924 12:50:58.435079 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.435098 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.435145 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:50:58.435162 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.435209 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:50:58.435226 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.438003 65034 executor.go:111] Tasks: 116 done / 124 total; 4 can run I0924 12:50:58.438067 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.438183 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.438300 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.438420 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.439680 65034 executor.go:111] Tasks: 120 done / 124 total; 2 can run I0924 12:50:58.439836 65034 executor.go:111] Tasks: 122 done / 124 total; 2 can run I0924 12:50:58.439903 65034 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0924 12:50:58.441826 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:50:58.441864 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:58.441892 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:50:58.441915 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-2"] }] } I0924 12:50:58.441939 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0924 12:50:58.441963 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:50:58.441984 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-2"] }] } I0924 12:50:58.442005 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:50:58.442026 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:50:58.442046 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:58.442068 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:50:58.442090 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:50:58.442113 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:50:58.442139 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0924 12:50:58.442160 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-2"] }] } I0924 12:50:58.442182 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:50:58.442203 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:50:58.442223 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:58.442246 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:50:58.442267 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:58.442289 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0924 12:50:58.442311 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:58.442333 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:58.442354 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:58.442377 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:58.442401 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:50:58.442422 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:58.442445 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:50:58.442468 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0924 12:50:58.442491 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-5"] }] } I0924 12:50:58.442514 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-6"] }] } I0924 12:50:58.442538 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:50:58.442558 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:50:58.442579 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:58.442603 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:58.442626 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:58.442649 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:50:58.443749 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:50:58.443787 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:50:58.443800 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:50:58.443827 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.443860 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0924 12:50:58.443897 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.443938 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.443969 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.444000 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.444029 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.444061 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0924 12:50:58.444102 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.444151 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:58.444171 65034 natgateway.go:117] DescribeNatGateways: { } I0924 12:50:58.444197 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0924 12:50:58.444222 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0924 12:50:58.444254 65034 api.go:53] PageSize not implemented I0924 12:50:58.444266 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:58.444282 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.444292 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0924 12:50:58.444316 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:50:58.444331 65034 targetgroups.go:35] PageSize not implemented I0924 12:50:58.444343 65034 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0924 12:50:58.444383 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:50:58.444407 65034 iamrole.go:89] ListRoles: { } I0924 12:50:58.444437 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.444456 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:58.444483 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:50:58.444508 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:58.444538 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-1","rtb-2"] } I0924 12:50:58.444564 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:50:58.444599 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-2"] } I0924 12:50:58.444627 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-2"] } I0924 12:50:58.444647 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0924 12:50:58.445028 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0924 12:50:58.445079 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0924 12:50:58.445105 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } autoscaling-config:lt-2 ok I0924 12:50:58.445153 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0924 12:50:58.445100 65034 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } autoscaling-group:nodes.minimal-ipv6.example.com ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0924 12:50:58.445186 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0924 12:50:58.445192 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:58.445221 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.445218 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0924 12:50:58.445257 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.445145 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:nodes.minimal-ipv6.example.com ok route53-record:Z1AFAKE1ZON3YO/AAAA/api.minimal-ipv6.example.com ok I0924 12:50:58.445234 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0924 12:50:58.445411 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:58.445425 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:50:58.445446 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:58.445478 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0924 12:50:58.445558 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:58.445567 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0924 12:50:58.445586 65034 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-2" } nat-gateway:nat-2 ok I0924 12:50:58.445619 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:50:58.445649 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0924 12:50:58.445685 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:50:58.445723 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0924 12:50:58.445744 65034 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0924 12:50:58.445773 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-6" } subnet:subnet-6 ok I0924 12:50:58.445804 65034 egressonlyinternetgateways.go:171] DeleteEgressOnlyInternetGateway: { EgressOnlyInternetGatewayId: "eigw-2" } egress-only-internet-gateway:eigw-2 ok I0924 12:50:58.445840 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:58.445867 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:50:58.445922 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:50:58.445953 65034 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0924 12:50:58.445945 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0924 12:50:58.445995 65034 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0924 12:50:58.445985 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0924 12:50:58.446007 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0924 12:50:58.446014 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0924 12:50:58.446029 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0924 12:50:58.446039 65034 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-2" } elastic-ip:eipalloc-2 ok I0924 12:50:58.446050 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.446074 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0924 12:50:58.446066 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-5" } subnet:subnet-5 ok I0924 12:50:58.446082 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0924 12:50:58.446098 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0924 12:50:58.446143 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:50:58.446177 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0924 12:50:58.446219 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:50:58.446262 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:50:58.446456 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.49s) === RUN TestLifecycleSharedVPC I0924 12:50:58.446591 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:58.446681 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:58.446707 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:58.446732 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:58.446757 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:58.446791 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:58.446818 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:58.446835 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:58.446858 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:58.446875 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:58.446904 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0924 12:50:58.461960 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:58.463632 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.463690 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.534361 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.596447 65034 iamrole.go:89] ListRoles: { } I0924 12:50:58.597261 65034 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0924 12:50:58.597528 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:58.597546 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0924 12:50:58.597601 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:50:58.597629 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:50:58.597614 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" }] } I0924 12:50:58.597669 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.598428 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0924 12:50:58.598490 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0924 12:50:58.598501 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.598647 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:58.598689 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:58.598811 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:58.599630 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:58.599842 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.600065 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:58.600871 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:58.603390 65034 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0924 12:50:58.603486 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0924 12:50:58.603511 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0924 12:50:58.603636 65034 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0924 12:50:58.603673 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0924 12:50:58.603587 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:58.603753 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0924 12:50:58.603901 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0924 12:50:58.603964 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.604130 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:50:58.604129 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0924 12:50:58.604200 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:58.604159 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0924 12:50:58.604240 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.604320 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:50:58.604408 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0924 12:50:58.604450 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:58.604498 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:58.604546 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:58.604600 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0924 12:50:58.604678 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:50:58.604760 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:58.604795 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:58.604821 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:58.605078 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0924 12:50:58.606308 65034 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0924 12:50:58.606438 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.606641 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.606723 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.606796 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0924 12:50:58.606966 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.607107 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0924 12:50:58.607200 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.607257 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.607337 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.607571 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I0924 12:50:58.607716 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I0924 12:50:58.607854 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.607943 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.608084 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.608157 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.608244 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:50:58.608285 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:50:58.608323 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.608504 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0924 12:50:58.608560 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0924 12:50:58.608613 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.608737 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" }] }] } I0924 12:50:58.608825 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0924 12:50:58.608911 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.609000 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0924 12:50:58.609124 65034 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0924 12:50:58.609305 65034 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0924 12:50:58.609397 65034 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0924 12:50:58.609426 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.609450 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:58.609490 65034 dns.go:232] Pre-creating DNS records I0924 12:50:58.609508 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:58.612693 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:50:58.614328 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.614377 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.686395 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.755354 65034 iamrole.go:89] ListRoles: { } I0924 12:50:58.756256 65034 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0924 12:50:58.756736 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:58.756674 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.757970 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.758153 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:50:58.758530 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:50:58.758625 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:50:58.758661 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:50:58.761653 65034 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0924 12:50:58.761757 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:50:58.761864 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0924 12:50:58.761963 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0924 12:50:58.762011 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:58.762696 65034 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0924 12:50:58.762096 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.762921 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.763118 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:58.763174 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0924 12:50:58.763372 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:58.765368 65034 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0924 12:50:58.765571 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.765753 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.765933 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.766030 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.766150 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.766279 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.766395 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.767698 65034 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0924 12:50:58.767862 65034 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0924 12:50:58.767985 65034 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0924 12:50:58.770098 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:50:58.770148 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:50:58.770178 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:50:58.770204 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:50:58.770232 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:50:58.770259 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:50:58.770291 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:50:58.770323 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:50:58.770355 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:50:58.770382 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:50:58.770407 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:58.770436 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:50:58.770462 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:58.770493 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:58.770520 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:58.770550 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:50:58.770577 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:50:58.770603 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:50:58.770632 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:58.770664 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:58.770695 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:50:58.771767 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:50:58.771820 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:50:58.771837 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:50:58.771870 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.771913 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0924 12:50:58.771961 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772010 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772044 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772078 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772112 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772146 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0924 12:50:58.772184 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772230 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0924 12:50:58.772260 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0924 12:50:58.772300 65034 api.go:53] PageSize not implemented I0924 12:50:58.772319 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:58.772340 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:58.772357 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:50:58.772377 65034 targetgroups.go:35] PageSize not implemented I0924 12:50:58.772393 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0924 12:50:58.772439 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:50:58.772468 65034 iamrole.go:89] ListRoles: { } I0924 12:50:58.772510 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:58.772535 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:58.772570 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:50:58.772601 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:50:58.772625 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:58.772933 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0924 12:50:58.772951 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok volume:vol-2 ok autoscaling-group:nodes.sharedvpc.example.com ok I0924 12:50:58.772991 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0924 12:50:58.773027 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } iam-instance-profile:nodes.sharedvpc.example.com ok I0924 12:50:58.773071 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0924 12:50:58.773113 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0924 12:50:58.773018 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0924 12:50:58.773088 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0924 12:50:58.773328 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:58.773353 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:50:58.773395 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0924 12:50:58.773453 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0924 12:50:58.773528 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:58.773576 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0924 12:50:58.773693 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:50:58.773707 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:50:58.773770 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0924 12:50:58.773822 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0924 12:50:58.773838 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0924 12:50:58.773873 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0924 12:50:58.773841 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:50:58.773906 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0924 12:50:58.773948 65034 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0924 12:50:58.773961 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0924 12:50:58.773991 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0924 12:50:58.774040 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:50:58.774284 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.33s) === RUN TestLifecycleComplex I0924 12:50:58.774447 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:58.774569 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:58.774611 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:58.774654 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:58.774693 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:58.774745 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:58.774786 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:58.774818 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:58.774856 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:58.774884 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:58.774932 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0924 12:50:58.812079 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:58.815978 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.816065 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:58.888984 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.005541 65034 iamrole.go:89] ListRoles: { } I0924 12:50:59.006763 65034 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0924 12:50:59.007075 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.007099 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } W0924 12:50:59.007232 65034 api.go:53] PageSize not implemented I0924 12:50:59.007210 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:59.007226 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.007264 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0924 12:50:59.007416 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:59.007568 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:50:59.007553 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0924 12:50:59.007610 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:50:59.007633 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0924 12:50:59.007690 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:50:59.007744 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0924 12:50:59.007825 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:50:59.007852 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:50:59.007907 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0924 12:50:59.008024 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0924 12:50:59.008078 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.008261 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:50:59.008420 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:50:59.008473 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:50:59.008579 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:50:59.008615 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:50:59.008644 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:59.008686 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.008827 65034 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0924 12:50:59.008959 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:50:59.008912 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:50:59.008928 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.008997 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0924 12:50:59.009020 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.009096 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:50:59.009091 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.009057 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:50:59.009164 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0924 12:50:59.009102 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Public" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0924 12:50:59.009169 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:50:59.009249 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0924 12:50:59.009334 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:50:59.009332 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0924 12:50:59.009444 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:59.009374 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0924 12:50:59.009491 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:59.009509 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:50:59.009439 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0924 12:50:59.009524 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.009621 65034 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0924 12:50:59.009669 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0924 12:50:59.009737 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:50:59.009815 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.009820 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0924 12:50:59.009913 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:50:59.009959 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0924 12:50:59.009982 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0924 12:50:59.010056 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.010135 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0924 12:50:59.010211 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:50:59.010224 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0924 12:50:59.010364 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" } ] }], VpcId: "vpc-1" } I0924 12:50:59.010446 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:50:59.010491 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:50:59.010365 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0924 12:50:59.010553 65034 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0924 12:50:59.010610 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.010676 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:59.010700 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:50:59.010779 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0924 12:50:59.010856 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:50:59.010888 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0924 12:50:59.010957 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:50:59.010985 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:59.011010 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.011139 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:59.011114 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0924 12:50:59.011184 65034 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0924 12:50:59.011228 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:59.011283 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.011350 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:50:59.011382 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:59.011417 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:50:59.011448 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:59.011479 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:50:59.011500 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:59.011560 65034 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0924 12:50:59.011665 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.011733 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.011777 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.011716 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0924 12:50:59.011845 65034 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0924 12:50:59.011864 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0924 12:50:59.011993 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0924 12:50:59.012049 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0924 12:50:59.012105 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0924 12:50:59.012160 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.012288 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0924 12:50:59.012418 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0924 12:50:59.012514 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.012609 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.012690 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0924 12:50:59.012714 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0924 12:50:59.012738 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.012808 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:50:59.012830 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:50:59.012889 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0924 12:50:59.012976 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0924 12:50:59.013014 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.013064 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.013099 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.013158 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.013230 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0924 12:50:59.013275 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0924 12:50:59.013306 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.013418 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:59.013464 65034 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0924 12:50:59.013524 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0924 12:50:59.013521 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0924 12:50:59.013568 65034 loadbalancers.go:170] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0924 12:50:59.013614 65034 api.go:53] PageSize not implemented I0924 12:50:59.013649 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0924 12:50:59.013758 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0924 12:50:59.013857 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0924 12:50:59.013919 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.014001 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0924 12:50:59.014083 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0924 12:50:59.014136 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0924 12:50:59.014167 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0924 12:50:59.014251 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.014336 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0924 12:50:59.014423 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.014506 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0924 12:50:59.014605 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:59.014640 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0924 12:50:59.014775 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.014878 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:50:59.014927 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0924 12:50:59.015063 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0924 12:50:59.015144 65034 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0924 12:50:59.015171 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.015217 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.015212 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:59.015241 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.015310 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.015357 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.015335 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:59.015369 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.018182 65034 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0924 12:50:59.018298 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.018374 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.018443 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.018517 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.020121 65034 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0924 12:50:59.020283 65034 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0924 12:50:59.020370 65034 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0924 12:50:59.020400 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.020423 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:59.020461 65034 dns.go:232] Pre-creating DNS records I0924 12:50:59.020479 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:59.024899 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:59.027135 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.027190 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.097076 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.235225 65034 iamrole.go:89] ListRoles: { } I0924 12:50:59.236509 65034 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0924 12:50:59.236776 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:50:59.236926 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.236918 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:59.237032 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:50:59.237128 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0924 12:50:59.237204 65034 api.go:53] PageSize not implemented I0924 12:50:59.237219 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:59.237247 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:59.237280 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.237435 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.237561 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:50:59.237667 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:50:59.237763 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.238799 65034 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0924 12:50:59.238959 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:50:59.238912 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:50:59.239024 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:50:59.239061 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:50:59.239122 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.239057 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.239158 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:50:59.239213 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:50:59.239297 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.239336 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:50:59.239215 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.239402 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:59.239464 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.239603 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.239695 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.239831 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.239923 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.240067 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:59.240105 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.240229 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:59.240282 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.240401 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:50:59.240566 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:59.240629 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.240739 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:59.240766 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:50:59.240855 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:59.240922 65034 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0924 12:50:59.241022 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.241120 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.241116 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.241200 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.241198 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.241220 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.241275 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.241323 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.241347 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.241370 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.241404 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.241437 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.241439 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.241491 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0924 12:50:59.241521 65034 api.go:53] PageSize not implemented I0924 12:50:59.241762 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.241824 65034 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0924 12:50:59.241863 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:59.241858 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:50:59.241907 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" W0924 12:50:59.241912 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.241929 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.242057 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.242088 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.242105 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.242190 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:50:59.242233 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.244850 65034 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0924 12:50:59.244943 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.245097 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.245273 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.245434 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.247788 65034 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0924 12:50:59.247998 65034 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0924 12:50:59.248125 65034 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0924 12:50:59.254412 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.257787 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.270964 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:59.274876 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.274940 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.346173 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.489232 65034 iamrole.go:89] ListRoles: { } I0924 12:50:59.490626 65034 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0924 12:50:59.490773 65034 api.go:53] PageSize not implemented I0924 12:50:59.490899 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:50:59.491038 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:50:59.491162 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:50:59.491244 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:59.491264 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.491395 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:50:59.491487 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.492877 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.493097 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.493236 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:59.493269 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:59.493350 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.494628 65034 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0924 12:50:59.494826 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:50:59.494761 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0924 12:50:59.494955 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:50:59.494903 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:50:59.495291 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.495332 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:50:59.495363 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:50:59.495410 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.495439 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:50:59.495555 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:50:59.495957 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0924 12:50:59.496834 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:59.496887 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.496998 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.497126 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.497998 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.498088 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.498148 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.498196 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.498237 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:50:59.498312 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.498373 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:50:59.498440 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:59.498463 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:59.498484 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:59.498535 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:59.498614 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:59.498667 65034 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0924 12:50:59.498851 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.499058 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.499048 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.499136 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.499160 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.499209 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.499248 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.499265 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.499349 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.499289 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.499423 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.499463 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.499497 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.499561 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.499621 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.499649 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0924 12:50:59.499684 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.499718 65034 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0924 12:50:59.499750 65034 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0924 12:50:59.499784 65034 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0924 12:50:59.499884 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0924 12:50:59.499956 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.499991 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } W0924 12:50:59.500032 65034 api.go:53] PageSize not implemented I0924 12:50:59.500089 65034 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0924 12:50:59.500154 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:59.500152 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:50:59.500230 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" W0924 12:50:59.500237 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.500257 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:50:59.500283 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.500339 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.500532 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.500585 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.500599 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.503668 65034 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0924 12:50:59.503733 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.503848 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.503983 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.504197 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.505819 65034 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0924 12:50:59.506030 65034 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0924 12:50:59.506134 65034 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0924 12:50:59.506177 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.506210 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:50:59.506274 65034 dns.go:232] Pre-creating DNS records I0924 12:50:59.513808 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:59.517760 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.517855 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.589115 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.731044 65034 iamrole.go:89] ListRoles: { } I0924 12:50:59.732330 65034 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0924 12:50:59.732468 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:50:59.733807 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0924 12:50:59.734061 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:50:59.734213 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:50:59.734343 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:50:59.734465 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.734605 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.734760 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:59.734880 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.734999 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W0924 12:50:59.735030 65034 api.go:53] PageSize not implemented I0924 12:50:59.735032 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:59.735110 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.737199 65034 executor.go:111] Tasks: 51 done / 122 total; 25 can run I0924 12:50:59.737733 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:50:59.737830 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:50:59.737820 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:50:59.737853 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.737908 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:50:59.737908 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.737965 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:50:59.737984 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.738038 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:50:59.738121 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.738160 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:50:59.738085 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:59.738228 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0924 12:50:59.738349 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:59.738376 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:50:59.738489 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:59.738573 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.738688 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:59.738716 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:50:59.738807 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:59.738837 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.738896 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.738940 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:59.738991 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.739070 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.739141 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.739189 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.739291 65034 executor.go:111] Tasks: 76 done / 122 total; 35 can run I0924 12:50:59.739373 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.739434 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.739449 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.739501 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.739513 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.739538 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.739560 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.739585 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.739606 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.739626 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.739587 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0924 12:50:59.739683 65034 api.go:53] PageSize not implemented I0924 12:50:59.739675 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.739763 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.739832 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.739960 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.740129 65034 executor.go:111] Tasks: 111 done / 122 total; 3 can run I0924 12:50:59.740154 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.740183 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.740189 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:59.740239 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0924 12:50:59.740193 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.740379 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.740408 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.740433 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.740551 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:50:59.740580 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.743407 65034 executor.go:111] Tasks: 114 done / 122 total; 4 can run I0924 12:50:59.743471 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.743575 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.743625 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.743734 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.745325 65034 executor.go:111] Tasks: 118 done / 122 total; 2 can run I0924 12:50:59.745544 65034 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0924 12:50:59.745655 65034 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0924 12:50:59.748968 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.752335 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.764208 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:50:59.768115 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.768210 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:50:59.840435 65034 zones.go:93] ListHostedZonesPages { } I0924 12:50:59.989907 65034 iamrole.go:89] ListRoles: { } I0924 12:50:59.991990 65034 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0924 12:50:59.992292 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:50:59.992405 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0924 12:50:59.992482 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0924 12:50:59.992603 65034 api.go:53] PageSize not implemented I0924 12:50:59.992599 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:59.992744 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.992869 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.992951 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:50:59.993010 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:50:59.993063 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.993141 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:59.993183 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.993206 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:50:59.993263 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:50:59.994686 65034 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0924 12:50:59.994827 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:50:59.994890 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:50:59.995078 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.995004 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:50:59.995119 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:50:59.995103 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:50:59.995188 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:50:59.995194 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:50:59.995152 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0924 12:50:59.995262 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.995293 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:50:59.995374 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:50:59.995434 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0924 12:50:59.995579 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:50:59.995737 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:50:59.995820 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.995914 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:50:59.996056 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.996198 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.996392 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:50:59.996451 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.996556 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:59.996661 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.996789 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.996928 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:50:59.997056 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:50:59.997106 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:59.997219 65034 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0924 12:50:59.997326 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.997397 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.997442 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.997497 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.997583 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.997557 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.997602 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.997650 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.997691 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.997734 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:50:59.997686 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:50:59.997773 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:50:59.997871 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:50:59.997893 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:50:59.997986 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0924 12:50:59.998037 65034 api.go:53] PageSize not implemented I0924 12:50:59.998200 65034 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0924 12:50:59.998230 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.998266 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.998278 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.998322 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:50:59.998410 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0924 12:50:59.998416 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:50:59.998498 65034 loadbalancers.go:35] PageSize not implemented I0924 12:50:59.998527 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:50:59.998608 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:50:59.998640 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.001161 65034 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0924 12:51:00.001292 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.001476 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.001644 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.001873 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.004976 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.005054 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.005279 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:51:00.005570 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:51:00.005719 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:51:00.005777 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.005822 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:00.005906 65034 dns.go:232] Pre-creating DNS records I0924 12:51:00.013301 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:51:00.017313 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.017407 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.088789 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.232240 65034 iamrole.go:89] ListRoles: { } I0924 12:51:00.233513 65034 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0924 12:51:00.233753 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.234020 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:00.234093 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:51:00.234159 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.234264 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:51:00.235088 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.235314 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.235367 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:51:00.235492 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:51:00.235610 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0924 12:51:00.236158 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0924 12:51:00.236243 65034 api.go:53] PageSize not implemented I0924 12:51:00.237198 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:00.237535 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:51:00.240976 65034 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0924 12:51:00.241124 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:51:00.241145 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.241235 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:51:00.241184 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:51:00.241306 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.241276 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.241353 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:51:00.241408 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:51:00.241449 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:51:00.241505 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.241522 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:00.241531 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:51:00.241560 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:00.241589 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.241673 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.241729 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.241799 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.241845 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.241898 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:51:00.241995 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:00.242030 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.242115 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:51:00.242139 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.242222 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:00.242246 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:51:00.242343 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:00.242400 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0924 12:51:00.242528 65034 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0924 12:51:00.242612 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.242754 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.242825 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:51:00.242880 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0924 12:51:00.242916 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.242941 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.242989 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.243026 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:00.243021 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.243065 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.243080 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.243104 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.243140 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:00.243131 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0924 12:51:00.243243 65034 api.go:53] PageSize not implemented I0924 12:51:00.243343 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.243412 65034 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0924 12:51:00.243436 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.243468 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.243479 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.243492 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:00.243588 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0924 12:51:00.243540 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.243624 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.243654 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.243734 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:51:00.243767 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.246381 65034 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0924 12:51:00.246446 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.246559 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.246650 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.246840 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.248644 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:51:00.248863 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:51:00.248989 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0924 12:51:00.253881 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.257216 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.257302 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.266745 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:51:00.270620 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.270712 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.270771 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.341896 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.489216 65034 iamrole.go:89] ListRoles: { } I0924 12:51:00.491352 65034 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0924 12:51:00.491578 65034 api.go:53] PageSize not implemented I0924 12:51:00.491615 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:00.491777 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0924 12:51:00.491973 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.492101 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.492180 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:51:00.492302 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.492415 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:51:00.492478 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:51:00.492534 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:51:00.492587 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0924 12:51:00.492645 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:00.492664 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:51:00.492700 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.494737 65034 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0924 12:51:00.494883 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.495022 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:51:00.495075 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:51:00.495173 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.495053 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:51:00.495225 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:51:00.495221 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:51:00.495304 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:51:00.495349 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:00.495376 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.495404 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:51:00.495433 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.495632 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:00.495678 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.495772 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.495859 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.495995 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:51:00.496150 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:00.496206 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.496332 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.496468 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.496612 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.496760 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:51:00.496900 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.497027 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:00.497092 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:00.497167 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:51:00.497241 65034 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0924 12:51:00.497388 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.497424 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.497528 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.497508 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.497561 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.497626 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.497664 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.497688 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:00.497762 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.497810 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.497845 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0924 12:51:00.497954 65034 api.go:53] PageSize not implemented I0924 12:51:00.497890 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.498175 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.498343 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.498661 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.498831 65034 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0924 12:51:00.498888 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.498967 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.498950 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:00.498999 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.499117 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.499113 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0924 12:51:00.499161 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.499353 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:51:00.499416 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.499491 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.504056 65034 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0924 12:51:00.504163 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.504332 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.504573 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.504794 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.507618 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:51:00.507945 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:51:00.508105 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:51:00.508155 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.508193 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:00.508255 65034 dns.go:232] Pre-creating DNS records I0924 12:51:00.515210 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:51:00.518712 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.518794 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.518843 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.590107 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.710443 65034 iamrole.go:89] ListRoles: { } I0924 12:51:00.711727 65034 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0924 12:51:00.711896 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:51:00.712017 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0924 12:51:00.712943 65034 api.go:53] PageSize not implemented I0924 12:51:00.712098 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0924 12:51:00.713157 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.713310 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:51:00.713409 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.713492 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:00.713522 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:51:00.713581 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.713740 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:51:00.714112 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.714534 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:51:00.712119 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:00.719647 65034 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0924 12:51:00.719884 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:51:00.719850 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.719972 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:51:00.719972 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:51:00.720070 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:51:00.720063 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.720108 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:00.720102 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:51:00.720152 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.720210 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.720240 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:51:00.720293 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.720308 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:51:00.720316 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:51:00.720336 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:51:00.720424 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:00.720466 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.720560 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:00.720585 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:51:00.720667 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:00.720694 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.720787 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:00.720811 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.720864 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.720948 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0924 12:51:00.721031 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.721081 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.721181 65034 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0924 12:51:00.721297 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.721363 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.721417 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.721371 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.721429 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.721455 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.721479 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:00.721509 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.721475 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.721531 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.721550 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0924 12:51:00.721611 65034 api.go:53] PageSize not implemented I0924 12:51:00.721672 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.721757 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.721879 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.722101 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.722252 65034 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0924 12:51:00.722278 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.722309 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.722316 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:00.722381 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0924 12:51:00.722335 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.722424 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.722458 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.722680 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.722865 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:51:00.722885 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.727520 65034 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0924 12:51:00.727654 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.727849 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.728016 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.728138 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.729825 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:51:00.730093 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:51:00.730248 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0924 12:51:00.732089 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.733970 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.734024 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.734103 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.739634 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:51:00.741853 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.741907 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.741940 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.741974 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.813596 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.982398 65034 iamrole.go:89] ListRoles: { } I0924 12:51:00.983660 65034 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0924 12:51:00.983800 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.983913 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:51:00.983986 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.984073 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:51:00.984063 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:00.984135 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0924 12:51:00.984190 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:00.984211 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W0924 12:51:00.984242 65034 api.go:53] PageSize not implemented I0924 12:51:00.984246 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.984327 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0924 12:51:00.984475 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0924 12:51:00.984579 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:51:00.984691 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0924 12:51:00.985851 65034 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0924 12:51:00.985965 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:51:00.986075 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:51:00.986106 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.986148 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.986189 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:51:00.986202 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:51:00.986171 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.986251 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:51:00.986259 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:51:00.986328 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.986293 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.986357 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:51:00.986435 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.986563 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:00.986597 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.986673 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:00.986697 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:51:00.986786 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:00.986812 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:51:00.986914 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:00.987006 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.987142 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:00.987223 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.987311 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.987437 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:00.987569 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:51:00.987607 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:00.987708 65034 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0924 12:51:00.987864 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.988009 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:51:00.988040 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0924 12:51:00.988080 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.988129 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.988164 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.988187 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:00.988179 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.988213 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:00.988237 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:00.988256 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0924 12:51:00.988311 65034 api.go:53] PageSize not implemented I0924 12:51:00.988337 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.988392 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.988438 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:00.988503 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:00.988570 65034 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0924 12:51:00.988600 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.988646 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.988610 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:00.988658 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.988698 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0924 12:51:00.988718 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:51:00.988738 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.988771 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:00.988816 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:00.988828 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:00.991399 65034 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0924 12:51:00.991459 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.991556 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.991621 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.991730 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:00.993239 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:51:00.993420 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:51:00.993511 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:51:00.993543 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:00.993566 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:00.993606 65034 dns.go:232] Pre-creating DNS records I0924 12:51:00.997978 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.9 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.9 ********************************************************************************* I0924 12:51:01.000213 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.000270 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.000307 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.000336 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.073901 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:01.199397 65034 iamrole.go:89] ListRoles: { } I0924 12:51:01.200916 65034 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0924 12:51:01.201125 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0924 12:51:01.201177 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:01.201224 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0924 12:51:01.201305 65034 api.go:53] PageSize not implemented I0924 12:51:01.201317 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.201426 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0924 12:51:01.203454 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.203679 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.204081 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0924 12:51:01.204192 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0924 12:51:01.204278 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:01.204320 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.204486 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:51:01.204566 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.207198 65034 executor.go:111] Tasks: 52 done / 123 total; 25 can run I0924 12:51:01.207335 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.207399 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0924 12:51:01.207404 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:51:01.207494 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:01.207440 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0924 12:51:01.207547 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0924 12:51:01.207614 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:01.207623 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0924 12:51:01.207696 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:01.207660 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0924 12:51:01.207732 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } I0924 12:51:01.207710 65034 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0924 12:51:01.207781 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:01.207875 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.207993 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0924 12:51:01.208084 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.208172 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.208267 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0924 12:51:01.208437 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:01.208493 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.208582 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0924 12:51:01.208745 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:01.208788 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.208897 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.209016 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.209164 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:01.209218 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0924 12:51:01.209301 65034 executor.go:111] Tasks: 77 done / 123 total; 35 can run I0924 12:51:01.209426 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:01.209516 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.209514 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:01.209706 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:01.209686 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:01.209730 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:01.209778 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:01.209791 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:01.209830 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:01.209874 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0924 12:51:01.209910 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0924 12:51:01.209914 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.209942 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } W0924 12:51:01.210039 65034 api.go:53] PageSize not implemented I0924 12:51:01.210067 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.210162 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.210234 65034 executor.go:111] Tasks: 112 done / 123 total; 3 can run I0924 12:51:01.210260 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:01.210292 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:01.210311 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:01.210317 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } I0924 12:51:01.210393 65034 dnsname.go:118] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" W0924 12:51:01.210406 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:01.210430 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:01.210620 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:01.210775 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { } I0924 12:51:01.210804 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:01.215635 65034 executor.go:111] Tasks: 115 done / 123 total; 4 can run I0924 12:51:01.215749 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.215926 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.216072 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.216210 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.218037 65034 executor.go:111] Tasks: 119 done / 123 total; 2 can run I0924 12:51:01.218271 65034 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0924 12:51:01.218389 65034 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0924 12:51:01.218502 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:51:01.218549 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:51:01.218584 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:51:01.218609 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:01.218640 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:51:01.218660 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:51:01.218682 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:51:01.218713 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:51:01.218736 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:51:01.218774 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:51:01.218794 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:51:01.218814 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:01.218837 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:51:01.218858 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:51:01.218888 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:01.218911 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:01.218934 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:51:01.218966 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:01.218992 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:01.219012 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:51:01.219035 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0924 12:51:01.219064 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:01.219084 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:51:01.219104 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:01.219137 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:01.219162 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:51:01.219202 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:51:01.220856 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:51:01.220903 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:51:01.220916 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:51:01.220941 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.220991 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0924 12:51:01.221041 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221090 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221119 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221163 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221190 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221228 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0924 12:51:01.221290 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221342 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0924 12:51:01.221376 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0924 12:51:01.221410 65034 api.go:53] PageSize not implemented I0924 12:51:01.221433 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:01.221449 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:01.221460 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0924 12:51:01.221487 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:51:01.221502 65034 targetgroups.go:35] PageSize not implemented I0924 12:51:01.221514 65034 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0924 12:51:01.221567 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:51:01.221593 65034 iamrole.go:89] ListRoles: { } I0924 12:51:01.221626 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:01.221644 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:01.221680 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:51:01.221710 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:51:01.221745 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0924 12:51:01.222107 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0924 12:51:01.222153 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0924 12:51:01.222189 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0924 12:51:01.222175 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:01.222225 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } autoscaling-group:nodes.complex.example.com ok I0924 12:51:01.222182 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } iam-instance-profile:masters.complex.example.com ok I0924 12:51:01.222243 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0924 12:51:01.222273 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0924 12:51:01.222287 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } security-group:sg-1 ok autoscaling-config:lt-2 ok subnet:subnet-3 ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0924 12:51:01.222339 65034 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0924 12:51:01.222348 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok volume:vol-1 ok I0924 12:51:01.222402 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:01.222307 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I0924 12:51:01.222467 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 } ] } W0924 12:51:01.222696 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:01.222710 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:51:01.222731 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:01.222760 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W0924 12:51:01.222905 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:01.222914 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0924 12:51:01.222933 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0924 12:51:01.222972 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:51:01.223012 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0924 12:51:01.223033 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok autoscaling-config:lt-1 ok I0924 12:51:01.223099 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0924 12:51:01.223137 65034 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0924 12:51:01.223147 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0924 12:51:01.223169 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0924 12:51:01.223189 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0924 12:51:01.223154 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0924 12:51:01.223214 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0924 12:51:01.223263 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0924 12:51:01.223254 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:51:01.223296 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0924 12:51:01.223343 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:51:01.223417 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:51:01.223468 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:51:01.223701 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.45s) === RUN TestLifecycleExternalLB I0924 12:51:01.223831 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:51:01.729662 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:51:01.729744 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:51:01.729804 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:51:01.729845 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:51:01.729915 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:51:01.729957 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:51:01.729987 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:51:01.730024 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:51:01.730050 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:51:01.730104 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0924 12:51:01.753202 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:01.755919 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.756008 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.826703 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:01.911060 65034 iamrole.go:89] ListRoles: { } I0924 12:51:01.913932 65034 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0924 12:51:01.914950 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:01.914942 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0924 12:51:01.915059 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0924 12:51:01.915033 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0924 12:51:01.915101 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } W0924 12:51:01.915026 65034 api.go:53] PageSize not implemented I0924 12:51:01.915157 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0924 12:51:01.915160 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:01.915190 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W0924 12:51:01.915255 65034 api.go:53] PageSize not implemented I0924 12:51:01.915291 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0924 12:51:01.915327 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } W0924 12:51:01.915356 65034 api.go:53] PageSize not implemented I0924 12:51:01.915344 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.915401 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:01.915463 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:01.915416 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0924 12:51:01.915464 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:51:01.915486 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0924 12:51:01.915519 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } I0924 12:51:01.915598 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:01.915734 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.915840 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:01.915949 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.915989 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] }] } I0924 12:51:01.916190 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:01.916355 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } I0924 12:51:01.916469 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:51:01.916506 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:51:01.916542 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:51:01.916587 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.916619 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:01.916747 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:01.916785 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:01.916856 65034 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0924 12:51:01.916943 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.917087 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:51:01.917124 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0924 12:51:01.917132 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:01.917268 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:51:01.917403 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0924 12:51:01.917426 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0924 12:51:01.917444 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0924 12:51:01.917535 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0924 12:51:01.917578 65034 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0924 12:51:01.917594 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }], VpcId: "vpc-1" } I0924 12:51:01.917659 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:01.917695 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:01.917675 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0924 12:51:01.917720 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:01.917800 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:51:01.917831 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:01.917955 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.918006 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:51:01.918051 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:01.918294 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0924 12:51:01.918339 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0924 12:51:01.918303 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0924 12:51:01.918620 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:01.918660 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] } I0924 12:51:01.918708 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:51:01.918768 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:01.918795 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:01.918832 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:01.918865 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:01.921973 65034 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0924 12:51:01.922102 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0924 12:51:01.922212 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.922320 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.922496 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:51:01.922548 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:51:01.922677 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.922964 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.923116 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.923224 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.923425 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.923616 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I0924 12:51:01.923839 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.923940 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0924 12:51:01.924111 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:01.924194 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:51:01.924247 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.924320 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:51:01.924377 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.924490 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.924579 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.924687 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.924769 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0924 12:51:01.924847 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0924 12:51:01.924933 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" }] }] } I0924 12:51:01.925011 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0924 12:51:01.925695 65034 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0924 12:51:01.926000 65034 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0924 12:51:01.926125 65034 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0924 12:51:01.926172 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:01.926213 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:01.926280 65034 dns.go:232] Pre-creating DNS records I0924 12:51:01.926311 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:01.932027 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:01.934886 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:01.934982 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.006426 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.101814 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.104029 65034 executor.go:111] Tasks: 0 done / 93 total; 49 can run I0924 12:51:02.104379 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } W0924 12:51:02.104550 65034 api.go:53] PageSize not implemented I0924 12:51:02.104424 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0924 12:51:02.104651 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0924 12:51:02.104698 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" } I0924 12:51:02.104794 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0924 12:51:02.104880 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0924 12:51:02.104798 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.104917 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" } I0924 12:51:02.105023 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:02.105003 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.105085 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:02.105113 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" } I0924 12:51:02.105201 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:02.105290 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W0924 12:51:02.105435 65034 api.go:53] PageSize not implemented I0924 12:51:02.105406 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:02.105595 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:02.105872 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.106259 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0924 12:51:02.106590 65034 api.go:53] PageSize not implemented I0924 12:51:02.108864 65034 executor.go:111] Tasks: 49 done / 93 total; 19 can run I0924 12:51:02.109055 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.109232 65034 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0924 12:51:02.109215 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.109373 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0924 12:51:02.109399 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.109573 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:02.109626 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:02.109724 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0924 12:51:02.109887 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.109948 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0924 12:51:02.110126 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.110199 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:02.113759 65034 executor.go:111] Tasks: 68 done / 93 total; 21 can run I0924 12:51:02.113977 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.114080 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:02.114185 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:02.114283 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:02.114340 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.114419 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.114696 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.117188 65034 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0924 12:51:02.117414 65034 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0924 12:51:02.117520 65034 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0924 12:51:02.120530 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:51:02.120589 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:51:02.120635 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:51:02.120671 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:02.120716 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:51:02.120754 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:51:02.120789 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:51:02.120838 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:51:02.120911 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:51:02.120989 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:51:02.121061 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:51:02.121126 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:51:02.121195 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:02.121280 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:51:02.121359 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:02.121424 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:02.121490 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:02.121559 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:02.121626 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:02.121689 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:51:02.121755 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:02.121829 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:02.121908 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:51:02.121980 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.124415 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:51:02.124507 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:51:02.124537 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:51:02.124626 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.124724 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0924 12:51:02.124832 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.124938 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.125024 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.125116 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.125192 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.125266 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0924 12:51:02.125348 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.125437 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0924 12:51:02.125509 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0924 12:51:02.125602 65034 api.go:53] PageSize not implemented I0924 12:51:02.125638 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:02.125688 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:02.125718 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:51:02.125762 65034 targetgroups.go:35] PageSize not implemented I0924 12:51:02.125794 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:02.125883 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:51:02.125950 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.126023 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.126070 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.126137 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:51:02.126206 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:51:02.126264 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0924 12:51:02.127100 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0924 12:51:02.127208 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0924 12:51:02.127234 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:51:02.127294 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0924 12:51:02.127332 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-group:nodes.externallb.example.com ok I0924 12:51:02.127407 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:51:02.127296 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.127451 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0924 12:51:02.127513 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.127535 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0924 12:51:02.127590 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0924 12:51:02.127740 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.127765 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:51:02.127808 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.127864 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0924 12:51:02.128118 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.128141 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:51:02.128184 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0924 12:51:02.128275 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:51:02.128340 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0924 12:51:02.128428 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0924 12:51:02.128448 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0924 12:51:02.128487 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0924 12:51:02.128526 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0924 12:51:02.128572 65034 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0924 12:51:02.128586 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0924 12:51:02.128615 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0924 12:51:02.128694 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:51:02.128778 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:51:02.128856 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:51:02.129142 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.91s) === RUN TestLifecycleSharedSubnet I0924 12:51:02.129362 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:51:02.129504 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:51:02.129544 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:51:02.129586 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:51:02.129626 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:51:02.129676 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:51:02.129720 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:51:02.129751 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:51:02.129790 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:51:02.129819 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:51:02.129899 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0924 12:51:02.154645 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:02.157573 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.157661 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.228904 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.321337 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.322851 65034 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0924 12:51:02.323070 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:51:02.323165 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:51:02.323187 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:51:02.323263 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0924 12:51:02.323307 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.323382 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I0924 12:51:02.323437 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:02.323553 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:02.323551 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:02.323621 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0924 12:51:02.323677 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.323759 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] } I0924 12:51:02.323853 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:02.323978 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:02.324016 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:02.324116 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.325248 65034 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0924 12:51:02.325407 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0924 12:51:02.325374 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0924 12:51:02.325520 65034 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0924 12:51:02.325558 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0924 12:51:02.325607 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0924 12:51:02.325674 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0924 12:51:02.325712 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:02.325740 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0924 12:51:02.325772 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.325820 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.325937 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0924 12:51:02.326128 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" }] }], VpcId: "vpc-12345678" } I0924 12:51:02.326234 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:02.326279 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.326340 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0924 12:51:02.326599 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:02.326664 65034 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" } ] } I0924 12:51:02.327719 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0924 12:51:02.328197 65034 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0924 12:51:02.328369 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.328478 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0924 12:51:02.328600 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.328736 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.328874 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I0924 12:51:02.329011 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.329072 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.329248 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.329384 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.329543 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0924 12:51:02.329752 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.329895 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.330032 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.330187 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" }] }] } I0924 12:51:02.330364 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" }] }] } I0924 12:51:02.330514 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.330650 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.330786 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0924 12:51:02.331792 65034 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0924 12:51:02.332039 65034 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0924 12:51:02.332168 65034 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0924 12:51:02.332213 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.332252 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.332319 65034 dns.go:232] Pre-creating DNS records I0924 12:51:02.332349 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.337697 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:02.340548 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.340639 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.412933 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.505659 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.507309 65034 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0924 12:51:02.507754 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:51:02.507880 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:51:02.507936 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:51:02.507954 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:02.507984 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] }] } I0924 12:51:02.508112 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.508191 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:02.510725 65034 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0924 12:51:02.510859 65034 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0924 12:51:02.510869 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0924 12:51:02.511100 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0924 12:51:02.511088 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.511176 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0924 12:51:02.511306 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.511365 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0924 12:51:02.511522 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.515616 65034 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0924 12:51:02.515855 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.516035 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.516336 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.516453 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.517798 65034 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0924 12:51:02.518056 65034 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0924 12:51:02.518199 65034 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0924 12:51:02.519711 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:51:02.519748 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:51:02.519768 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:02.519787 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:51:02.519806 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:51:02.519824 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:51:02.519845 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:51:02.519866 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:51:02.519892 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:51:02.519916 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:51:02.519934 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:51:02.519956 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:02.519977 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:02.519999 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:02.520020 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:02.520043 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:51:02.520064 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:02.520087 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:02.520112 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.520877 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:51:02.520916 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:51:02.520925 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:51:02.520946 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.520973 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0924 12:51:02.521004 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521041 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521068 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521092 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521116 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521138 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0924 12:51:02.521159 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521185 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0924 12:51:02.521204 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0924 12:51:02.521231 65034 api.go:53] PageSize not implemented I0924 12:51:02.521244 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:02.521279 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:02.521292 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:51:02.521306 65034 targetgroups.go:35] PageSize not implemented I0924 12:51:02.521319 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:02.521354 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:51:02.521377 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.521406 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.521426 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.521451 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:51:02.521474 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:51:02.521671 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0924 12:51:02.521713 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok autoscaling-group:nodes.sharedsubnet.example.com ok I0924 12:51:02.521740 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.521779 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0924 12:51:02.521896 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0924 12:51:02.521815 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:nodes.sharedsubnet.example.com ok volume:vol-2 ok I0924 12:51:02.521945 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0924 12:51:02.521993 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.522028 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0924 12:51:02.522111 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.522121 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0924 12:51:02.522157 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.522185 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0924 12:51:02.522289 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.522300 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:51:02.522323 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:51:02.522366 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0924 12:51:02.522396 65034 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0924 12:51:02.522406 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0924 12:51:02.522427 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0924 12:51:02.522451 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0924 12:51:02.522477 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0924 12:51:02.522485 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0924 12:51:02.522501 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0924 12:51:02.522645 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedSubnet I0924 12:51:02.522757 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:51:02.522839 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:51:02.522863 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:51:02.522889 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:51:02.522912 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:51:02.522945 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:51:02.522972 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:51:02.522989 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:51:02.523012 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:51:02.523029 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:51:02.523059 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0924 12:51:02.539119 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:02.541281 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.541330 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.541365 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.611962 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.699582 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.702020 65034 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0924 12:51:02.702301 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:02.702348 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.702678 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:02.702928 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:02.703003 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.703174 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0924 12:51:02.703225 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.703598 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0924 12:51:02.703809 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] } I0924 12:51:02.703762 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:51:02.704228 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:51:02.704277 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:51:02.704184 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0924 12:51:02.704413 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:02.705134 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0924 12:51:02.705338 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:02.706073 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:02.706152 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.707755 65034 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0924 12:51:02.707946 65034 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0924 12:51:02.707878 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0924 12:51:02.708044 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.708025 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0924 12:51:02.708110 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0924 12:51:02.708165 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0924 12:51:02.708221 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0924 12:51:02.708243 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-su-5ol32q", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-su-5ol32q" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0924 12:51:02.708304 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0924 12:51:02.708344 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.708442 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0924 12:51:02.708393 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.timeout_seconds", Value: "30" },{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0924 12:51:02.708452 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:02.708512 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:02.708545 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.708571 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0924 12:51:02.708720 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0924 12:51:02.708681 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0924 12:51:02.708773 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0924 12:51:02.708869 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:02.708883 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.708930 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:02.708958 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.708980 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0924 12:51:02.709085 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:02.709138 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:51:02.709164 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:02.709194 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:02.709219 65034 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" } ] } I0924 12:51:02.709264 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:02.709287 65034 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" } ] } I0924 12:51:02.709350 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:02.709468 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:51:02.709527 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:51:02.710633 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0924 12:51:02.711126 65034 executor.go:111] Tasks: 67 done / 107 total; 27 can run W0924 12:51:02.711227 65034 api.go:53] PageSize not implemented I0924 12:51:02.711271 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I0924 12:51:02.711393 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.711423 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:02.711489 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:02.711439 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0924 12:51:02.711554 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.711587 65034 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-su-5ol32q", SubnetMappings: [{ SubnetId: "subnet-abcdef" }], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" }], Type: "network" } I0924 12:51:02.711484 65034 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0924 12:51:02.711672 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0924 12:51:02.711737 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0924 12:51:02.711718 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.711777 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1", Port: 22, Protocol: "TCP" } I0924 12:51:02.711784 65034 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } W0924 12:51:02.711844 65034 api.go:53] PageSize not implemented I0924 12:51:02.711860 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.711886 65034 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0924 12:51:02.711911 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0924 12:51:02.712133 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0924 12:51:02.712359 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.712512 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.712681 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0924 12:51:02.712836 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:51:02.712929 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.713088 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0924 12:51:02.713191 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.713354 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.713446 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.713521 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0924 12:51:02.713741 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:51:02.713850 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" }] }] } I0924 12:51:02.713988 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.714146 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" }] }] } I0924 12:51:02.714288 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.714425 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.714554 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.714713 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0924 12:51:02.714854 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.715002 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0924 12:51:02.715203 65034 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0924 12:51:02.715297 65034 api.go:53] PageSize not implemented I0924 12:51:02.715283 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:02.715341 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.715433 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } W0924 12:51:02.715511 65034 api.go:53] PageSize not implemented I0924 12:51:02.715577 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.720328 65034 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0924 12:51:02.720576 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.720677 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.720786 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.720913 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.723215 65034 executor.go:111] Tasks: 103 done / 107 total; 2 can run I0924 12:51:02.723482 65034 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0924 12:51:02.723605 65034 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0924 12:51:02.723644 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.723677 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.723738 65034 dns.go:232] Pre-creating DNS records I0924 12:51:02.723763 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.730304 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:02.733343 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.733419 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.733482 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.806942 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.872704 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.877663 65034 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0924 12:51:02.878005 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:51:02.878054 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:02.878151 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:51:02.878193 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:51:02.878305 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.878484 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.878628 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:02.880368 65034 executor.go:111] Tasks: 45 done / 107 total; 22 can run I0924 12:51:02.880703 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0924 12:51:02.880730 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0924 12:51:02.880760 65034 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0924 12:51:02.880658 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0924 12:51:02.880814 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-su-5ol32q"] } I0924 12:51:02.880828 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0924 12:51:02.880952 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:02.880878 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0924 12:51:02.880989 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.880989 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } I0924 12:51:02.881019 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0924 12:51:02.881108 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:51:02.881136 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0924 12:51:02.881214 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:02.881261 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.881332 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0924 12:51:02.881398 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0924 12:51:02.882250 65034 executor.go:111] Tasks: 67 done / 107 total; 27 can run W0924 12:51:02.882365 65034 api.go:53] PageSize not implemented I0924 12:51:02.882386 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.882443 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.882469 65034 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0924 12:51:02.882369 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.882481 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:02.882575 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:02.882597 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0924 12:51:02.882625 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.882653 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0924 12:51:02.882745 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } I0924 12:51:02.882792 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4"] } I0924 12:51:02.882831 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } W0924 12:51:02.882916 65034 api.go:53] PageSize not implemented I0924 12:51:02.882939 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.883137 65034 executor.go:111] Tasks: 94 done / 107 total; 4 can run W0924 12:51:02.883187 65034 api.go:53] PageSize not implemented I0924 12:51:02.883221 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.883211 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:02.883293 65034 dnsname.go:118] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" W0924 12:51:02.883327 65034 api.go:53] PageSize not implemented I0924 12:51:02.883390 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.883564 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.889008 65034 executor.go:111] Tasks: 98 done / 107 total; 5 can run I0924 12:51:02.889160 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.889280 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.889362 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.889551 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.892031 65034 executor.go:111] Tasks: 103 done / 107 total; 2 can run W0924 12:51:02.892132 65034 api.go:53] PageSize not implemented I0924 12:51:02.892149 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.892277 65034 executor.go:111] Tasks: 105 done / 107 total; 2 can run I0924 12:51:02.892443 65034 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0924 12:51:02.895696 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:51:02.895761 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:51:02.895799 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:02.895841 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:51:02.895875 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:51:02.895911 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:51:02.895949 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:51:02.895990 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:51:02.896031 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0924 12:51:02.896075 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:51:02.896110 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:51:02.896143 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:51:02.896177 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:02.896213 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:02.896249 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:51:02.896286 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:51:02.896321 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:02.896360 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:02.896399 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:51:02.896435 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:02.896473 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:02.896510 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:51:02.898068 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:51:02.898136 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:51:02.898155 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:51:02.898197 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898258 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0924 12:51:02.898325 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898389 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898434 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898478 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898523 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898560 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0924 12:51:02.898601 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898646 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0924 12:51:02.898680 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0924 12:51:02.898725 65034 api.go:53] PageSize not implemented I0924 12:51:02.898743 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0924 12:51:02.898782 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:02.898811 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:02.898828 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1"] } I0924 12:51:02.898866 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:51:02.898890 65034 targetgroups.go:35] PageSize not implemented I0924 12:51:02.898907 65034 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" ] } I0924 12:51:02.898973 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:51:02.899013 65034 iamrole.go:89] ListRoles: { } I0924 12:51:02.899063 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:02.899093 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:02.899133 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:51:02.899179 65034 routetable.go:53] DescribeRouteTables: { } autoscaling-group:bastion.private-shared-subnet.example.com ok autoscaling-config:lt-3 ok I0924 12:51:02.899700 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-su-5ol32q/4 ok I0924 12:51:02.899702 65034 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I0924 12:51:02.899732 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0924 12:51:02.899772 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.899807 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.899848 65034 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-su-5ol32q/1 ok I0924 12:51:02.899798 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } autoscaling-group:nodes.private-shared-subnet.example.com ok W0924 12:51:02.899965 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.899985 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:51:02.899982 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0924 12:51:02.900014 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0924 12:51:02.899844 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:bastions.private-shared-subnet.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok I0924 12:51:02.900021 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:51:02.900039 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } autoscaling-config:lt-2 ok I0924 12:51:02.900134 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0924 12:51:02.900127 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0924 12:51:02.900205 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0924 12:51:02.900273 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:51:02.900335 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 } ] } W0924 12:51:02.900501 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.900524 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0924 12:51:02.900561 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:02.900616 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0924 12:51:02.900737 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.900758 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-1 ok I0924 12:51:02.900832 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:02.900855 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0924 12:51:02.900949 65034 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0924 12:51:02.900881 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0924 12:51:02.900980 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:02.900970 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0924 12:51:02.900996 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:51:02.901008 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0924 12:51:02.901044 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0924 12:51:02.901096 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0924 12:51:02.901112 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0924 12:51:02.901143 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0924 12:51:02.901172 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.901226 65034 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0924 12:51:02.901243 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0924 12:51:02.901304 65034 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0924 12:51:02.901561 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I0924 12:51:02.901768 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:51:02.901908 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:51:02.901949 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:51:02.901994 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:51:02.902034 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:51:02.902084 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:51:02.902129 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:51:02.902160 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:51:02.902199 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:51:02.902239 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:51:02.902290 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0924 12:51:02.930710 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:02.933827 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.933936 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:02.934007 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.004504 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:03.079640 65034 iamrole.go:89] ListRoles: { } I0924 12:51:03.081689 65034 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0924 12:51:03.082056 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:03.082268 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:51:03.082114 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0924 12:51:03.082376 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:51:03.082421 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:51:03.082440 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:03.082403 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0924 12:51:03.082471 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.082666 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.082814 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] }] } I0924 12:51:03.082826 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0924 12:51:03.082946 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:03.082963 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0924 12:51:03.083038 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:03.083087 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0924 12:51:03.083207 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0924 12:51:03.083130 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:03.083379 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:03.084216 65034 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0924 12:51:03.084365 65034 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0924 12:51:03.084400 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0924 12:51:03.084513 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0924 12:51:03.084499 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.084559 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0924 12:51:03.084617 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0924 12:51:03.084611 65034 targetgroups.go:94] CreateTargetGroup { HealthCheckIntervalSeconds: 10, HealthyThresholdCount: 2, Name: "bastion-private-shared-ip-eepmph", Port: 22, Protocol: "TCP", Tags: [{ Key: "Name", Value: "bastion-private-shared-ip-eepmph" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-12345678" } I0924 12:51:03.084733 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0924 12:51:03.084743 65034 targetgroups.go:149] ModifyTargetGroupAttributes { Attributes: [{ Key: "deregistration_delay.connection_termination.enabled", Value: "true" },{ Key: "deregistration_delay.timeout_seconds", Value: "30" }], TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0924 12:51:03.084759 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:03.084815 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:03.084846 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:03.084795 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0924 12:51:03.084871 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0924 12:51:03.084888 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0924 12:51:03.084950 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0924 12:51:03.085008 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0924 12:51:03.085000 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.085154 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0924 12:51:03.085204 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0924 12:51:03.085234 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0924 12:51:03.085334 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:03.085396 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:03.085426 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:03.085460 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:03.085492 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:03.085515 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.085626 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-12345678" } I0924 12:51:03.085668 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:51:03.085822 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0924 12:51:03.086054 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0924 12:51:03.086160 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0924 12:51:03.086288 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.086411 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:51:03.086469 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:03.086513 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:51:03.086565 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:03.086607 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:03.086369 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0924 12:51:03.086661 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" } ] } I0924 12:51:03.086744 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0924 12:51:03.086844 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:51:03.086887 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:51:03.086926 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:51:03.087067 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0924 12:51:03.087158 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:51:03.087206 65034 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0924 12:51:03.087325 65034 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0924 12:51:03.087410 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0924 12:51:03.087501 65034 api.go:53] PageSize not implemented I0924 12:51:03.087503 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:03.087555 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:03.087547 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.087589 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.087642 65034 loadbalancers.go:81] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "bastion-private-shared-ip-eepmph", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }], Type: "network" } I0924 12:51:03.087659 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0924 12:51:03.087763 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0924 12:51:03.087709 65034 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0924 12:51:03.087802 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.087845 65034 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0924 12:51:03.087792 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:03.087910 65034 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0924 12:51:03.088045 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:51:03.088117 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0924 12:51:03.088236 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.088491 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.088643 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.088863 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.089016 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:03.089092 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0924 12:51:03.089179 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" }] }] } I0924 12:51:03.089365 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-172.20.4.0/22-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.089511 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.089651 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:03.089720 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:51:03.089852 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" }] }] } I0924 12:51:03.089952 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:03.089981 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0924 12:51:03.090012 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.090108 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:03.090084 65034 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1", Port: 22, Protocol: "TCP" } I0924 12:51:03.090194 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } W0924 12:51:03.090208 65034 api.go:53] PageSize not implemented I0924 12:51:03.090231 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.090230 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.090394 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.090543 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0924 12:51:03.090640 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.090777 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.090908 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.091003 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.091091 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 }] } I0924 12:51:03.091199 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.091356 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.091490 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.091620 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0924 12:51:03.091671 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.091800 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.091932 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0924 12:51:03.092037 65034 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0924 12:51:03.092091 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0924 12:51:03.092257 65034 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0924 12:51:03.092292 65034 api.go:53] PageSize not implemented I0924 12:51:03.092337 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:03.092356 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:03.092335 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.092436 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:03.092462 65034 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } W0924 12:51:03.092555 65034 api.go:53] PageSize not implemented I0924 12:51:03.092594 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.092590 65034 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0924 12:51:03.092694 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0924 12:51:03.092739 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:03.092781 65034 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0924 12:51:03.095228 65034 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0924 12:51:03.095281 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:03.095362 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0924 12:51:03.095447 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.095513 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.095597 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.095698 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.098055 65034 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0924 12:51:03.098285 65034 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0924 12:51:03.098376 65034 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0924 12:51:03.098408 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:03.098430 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:03.098470 65034 dns.go:232] Pre-creating DNS records I0924 12:51:03.098487 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:03.101863 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:03.103613 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.103661 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.103695 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.173426 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:03.247588 65034 iamrole.go:89] ListRoles: { } I0924 12:51:03.249068 65034 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0924 12:51:03.249461 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.249611 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:03.249640 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I0924 12:51:03.249719 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0924 12:51:03.249764 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0924 12:51:03.249786 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0924 12:51:03.249813 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:03.251237 65034 executor.go:111] Tasks: 45 done / 116 total; 24 can run I0924 12:51:03.251338 65034 targetgroups.go:32] DescribeTargetGroups { Names: ["bastion-private-shared-ip-eepmph"] } I0924 12:51:03.251417 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0924 12:51:03.251454 65034 targetgroups.go:139] DescribeTargetGroupAttributes { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } I0924 12:51:03.251486 65034 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0924 12:51:03.251401 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0924 12:51:03.251528 65034 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0924 12:51:03.251571 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0924 12:51:03.251559 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0924 12:51:03.251673 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:03.251712 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:03.251749 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0924 12:51:03.251812 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.251906 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0924 12:51:03.251930 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.252007 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:03.252032 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.252138 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0924 12:51:03.252240 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:03.252273 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.252347 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0924 12:51:03.252432 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:51:03.252525 65034 executor.go:111] Tasks: 69 done / 116 total; 32 can run I0924 12:51:03.252700 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } W0924 12:51:03.252813 65034 api.go:53] PageSize not implemented I0924 12:51:03.252798 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:03.252861 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:03.252874 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0924 12:51:03.252847 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:03.252911 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0924 12:51:03.252938 65034 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0924 12:51:03.252963 65034 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4"] } I0924 12:51:03.252941 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0924 12:51:03.252988 65034 loadbalancers.go:156] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } I0924 12:51:03.253004 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:03.252842 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.253053 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:03.253103 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.253148 65034 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W0924 12:51:03.253263 65034 api.go:53] PageSize not implemented I0924 12:51:03.253284 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.253287 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:03.253424 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0924 12:51:03.253613 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.253822 65034 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0924 12:51:03.253866 65034 api.go:53] PageSize not implemented I0924 12:51:03.253898 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0924 12:51:03.253915 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.253920 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:03.253971 65034 dnsname.go:118] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" W0924 12:51:03.253999 65034 api.go:53] PageSize not implemented I0924 12:51:03.253995 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:51:03.254052 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.254219 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.258523 65034 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0924 12:51:03.258605 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:03.258720 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.258917 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.259065 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.259180 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.260486 65034 executor.go:111] Tasks: 112 done / 116 total; 2 can run W0924 12:51:03.260583 65034 api.go:53] PageSize not implemented I0924 12:51:03.260604 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.260741 65034 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0924 12:51:03.260849 65034 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0924 12:51:03.263464 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:51:03.263535 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:51:03.263573 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:03.263612 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:51:03.263636 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:51:03.263674 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:51:03.263699 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:51:03.263741 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:51:03.263790 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0924 12:51:03.263820 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0924 12:51:03.263859 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:51:03.263888 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:51:03.263915 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:03.263946 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:51:03.263974 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0924 12:51:03.264003 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:03.264043 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:03.264070 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0924 12:51:03.264111 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0924 12:51:03.264145 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:03.264179 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:03.264207 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0924 12:51:03.264239 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:03.264267 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:51:03.264294 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:03.264326 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:03.264359 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:51:03.265594 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:51:03.265653 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:51:03.265670 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:51:03.265706 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.265757 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0924 12:51:03.265814 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.265869 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.265904 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.265938 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.265975 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.266012 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0924 12:51:03.266056 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.266102 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:51:03.266125 65034 natgateway.go:117] DescribeNatGateways: { } I0924 12:51:03.266159 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0924 12:51:03.266189 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0924 12:51:03.266227 65034 api.go:53] PageSize not implemented I0924 12:51:03.266240 65034 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0924 12:51:03.266272 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:03.266292 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:03.266304 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1"] } I0924 12:51:03.266337 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:51:03.266356 65034 targetgroups.go:35] PageSize not implemented I0924 12:51:03.266370 65034 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" ] } I0924 12:51:03.266423 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:51:03.266456 65034 iamrole.go:89] ListRoles: { } I0924 12:51:03.266502 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:03.266527 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:03.266564 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:51:03.266600 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:51:03.266629 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0924 12:51:03.266658 65034 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0924 12:51:03.266686 65034 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0924 12:51:03.267139 65034 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/bastion-private-shared-ip-eepmph/4 ok I0924 12:51:03.267184 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0924 12:51:03.267177 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0924 12:51:03.267204 65034 loadbalancers.go:208] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1" } autoscaling-group:nodes.private-shared-ip.example.com ok I0924 12:51:03.267250 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok autoscaling-group:bastion.private-shared-ip.example.com ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/bastion-private-shared-ip-eepmph/1 ok I0924 12:51:03.267293 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0924 12:51:03.267242 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0924 12:51:03.267384 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:03.267385 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0924 12:51:03.267396 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } I0924 12:51:03.267266 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } security-group:sg-3 ok route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok autoscaling-config:lt-3 ok I0924 12:51:03.267363 65034 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok I0924 12:51:03.267441 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0924 12:51:03.267521 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0924 12:51:03.267456 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0924 12:51:03.267577 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "172.20.4.0/22" }], ToPort: 4 } ] } W0924 12:51:03.267735 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:03.267753 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0924 12:51:03.267780 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:51:03.267830 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0924 12:51:03.267888 65034 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0924 12:51:03.267958 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0924 12:51:03.268000 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:03.268049 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0924 12:51:03.268145 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:03.268157 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0924 12:51:03.268219 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0924 12:51:03.268262 65034 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0924 12:51:03.268276 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0924 12:51:03.268260 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0924 12:51:03.268313 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0924 12:51:03.268331 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:03.268384 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0924 12:51:03.268402 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0924 12:51:03.268477 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:03.268486 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } I0924 12:51:03.268477 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } security-group:sg-2 ok I0924 12:51:03.268511 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } I0924 12:51:03.268510 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } iam-role:nodes.private-shared-ip.example.com ok subnet:subnet-1 ok I0924 12:51:03.268561 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0924 12:51:03.268602 65034 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0924 12:51:03.268616 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0924 12:51:03.268640 65034 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0924 12:51:03.268666 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0924 12:51:03.268716 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:51:03.268753 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0924 12:51:03.268927 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.37s) === RUN TestLifecycleManyAddons I0924 12:51:03.269069 65034 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:51:03.269153 65034 internetgateways.go:66] CreateInternetGateway: { } I0924 12:51:03.269179 65034 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:51:03.269205 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:51:03.269228 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:51:03.269261 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:51:03.269292 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:51:03.269312 65034 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:51:03.269336 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:51:03.269362 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:51:03.269394 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0924 12:51:03.285842 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:03.288518 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.288589 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:03.360816 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:04.169297 65034 iamrole.go:89] ListRoles: { } I0924 12:51:04.170917 65034 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0924 12:51:04.171569 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0924 12:51:04.171737 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0924 12:51:04.171786 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.171853 65034 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0924 12:51:04.171912 65034 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0924 12:51:04.171849 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:04.171972 65034 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.172112 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:51:04.172202 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:04.172381 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.172598 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.172738 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:04.172823 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:04.172856 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.172954 65034 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.173019 65034 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.173277 65034 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/control-plane", Value: "1" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0924 12:51:04.173446 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:04.173519 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0924 12:51:04.173557 65034 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0924 12:51:04.173587 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:51:04.173635 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:04.175772 65034 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0924 12:51:04.176134 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0924 12:51:04.176193 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:51:04.176297 65034 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:51:04.176283 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.176388 65034 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0924 12:51:04.176410 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0924 12:51:04.176476 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0924 12:51:04.176544 65034 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0924 12:51:04.176527 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeTags\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:51:04.176603 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:51:04.176668 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:04.176704 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.176905 65034 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0924 12:51:04.177004 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:51:04.177063 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:04.177264 65034 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" } ] }], VpcId: "vpc-1" } I0924 12:51:04.177321 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0924 12:51:04.177532 65034 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0924 12:51:04.177605 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:04.177668 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:04.177735 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:04.177791 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:04.177881 65034 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0924 12:51:04.177974 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:04.178069 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:04.178124 65034 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I0924 12:51:04.178217 65034 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:51:04.178266 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:04.178329 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:04.178625 65034 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:network-interface/*\"\n ]\n },\n {\n \"Action\": [\n \"acm:DescribeCertificate\",\n \"acm:ListCertificates\",\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignPrivateIpAddresses\",\n \"ec2:AttachNetworkInterface\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateNetworkInterface\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateTags\",\n \"ec2:DeleteNetworkInterface\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeInternetGateways\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSnapshots\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcPeeringConnections\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachNetworkInterface\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:UnassignPrivateIpAddresses\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListenerCertificates\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeRules\",\n \"elasticloadbalancing:DescribeTags\",\n \"elasticloadbalancing:DescribeTargetGroupAttributes\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteSnapshot\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddListenerCertificates\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteRule\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyRule\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:ModifyTargetGroupAttributes\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:RemoveListenerCertificates\",\n \"elasticloadbalancing:RemoveTags\",\n \"elasticloadbalancing:SetIpAddressType\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\",\n \"elasticloadbalancing:SetSecurityGroups\",\n \"elasticloadbalancing:SetSubnets\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateRule\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:51:04.180987 65034 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0924 12:51:04.181221 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.181435 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0924 12:51:04.181581 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:04.181723 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0924 12:51:04.181775 65034 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0924 12:51:04.181892 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.182046 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:04.182196 65034 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:51:04.182324 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" }] }] } I0924 12:51:04.182540 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.182685 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:04.182773 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" }] }] } I0924 12:51:04.182915 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0924 12:51:04.183049 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I0924 12:51:04.183239 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0924 12:51:04.183399 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" }] }] } I0924 12:51:04.183544 65034 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0924 12:51:04.183593 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.183735 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.183868 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.184009 65034 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" }] }] } I0924 12:51:04.184154 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:04.184259 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:04.184350 65034 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0924 12:51:04.184559 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:04.184690 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:04.187334 65034 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0924 12:51:04.187678 65034 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0924 12:51:04.187908 65034 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0924 12:51:04.187979 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:04.188024 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:04.188096 65034 dns.go:232] Pre-creating DNS records I0924 12:51:04.188127 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:04.197860 65034 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.25.3 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.25.3 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0924 12:51:04.202303 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:04.202406 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:04.275315 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:05.021197 65034 iamrole.go:89] ListRoles: { } I0924 12:51:05.022261 65034 executor.go:111] Tasks: 0 done / 107 total; 52 can run I0924 12:51:05.022671 65034 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0924 12:51:05.022804 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.022977 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.026579 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.026819 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:51:05.026854 65034 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:51:05.026903 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.027040 65034 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0924 12:51:05.027213 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:05.030254 65034 executor.go:111] Tasks: 52 done / 107 total; 23 can run I0924 12:51:05.030399 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.030499 65034 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0924 12:51:05.030613 65034 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0924 12:51:05.030620 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0924 12:51:05.030839 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.030994 65034 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:51:05.031041 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.031170 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:05.031296 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0924 12:51:05.031448 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:05.031504 65034 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:51:05.031622 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:05.035814 65034 executor.go:111] Tasks: 75 done / 107 total; 26 can run I0924 12:51:05.036055 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:05.036172 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:05.036478 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:05.036647 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:05.036738 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0924 12:51:05.036945 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:05.037121 65034 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: ["images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0924 12:51:05.040125 65034 executor.go:111] Tasks: 101 done / 107 total; 2 can run I0924 12:51:05.040461 65034 executor.go:111] Tasks: 103 done / 107 total; 4 can run I0924 12:51:05.040683 65034 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0924 12:51:05.043638 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0924 12:51:05.043700 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0924 12:51:05.043740 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eigw-1"] }] } I0924 12:51:05.043773 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0924 12:51:05.043808 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0924 12:51:05.043841 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0924 12:51:05.043875 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0924 12:51:05.043911 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0924 12:51:05.043948 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0924 12:51:05.043987 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0924 12:51:05.044030 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0924 12:51:05.044066 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0924 12:51:05.044097 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0924 12:51:05.044133 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0924 12:51:05.044164 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:51:05.044199 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0924 12:51:05.044234 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:51:05.044272 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0924 12:51:05.044307 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0924 12:51:05.044342 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0924 12:51:05.044376 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0924 12:51:05.044413 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0924 12:51:05.044451 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:51:05.044486 65034 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } I0924 12:51:05.046154 65034 delete_cluster.go:128] Looking for cloud resources to delete W0924 12:51:05.046217 65034 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0924 12:51:05.046238 65034 keypairs.go:95] DescribeKeyPairs: { } I0924 12:51:05.046281 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046333 65034 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0924 12:51:05.046389 65034 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046449 65034 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046497 65034 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046552 65034 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046595 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046636 65034 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0924 12:51:05.046694 65034 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046746 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0924 12:51:05.046786 65034 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0924 12:51:05.046832 65034 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0924 12:51:05.046870 65034 api.go:53] PageSize not implemented I0924 12:51:05.046891 65034 loadbalancers.go:32] DescribeLoadBalancers v2 { PageSize: 20 } W0924 12:51:05.046919 65034 loadbalancers.go:35] PageSize not implemented I0924 12:51:05.046938 65034 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0924 12:51:05.046970 65034 targetgroups.go:35] PageSize not implemented I0924 12:51:05.046991 65034 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0924 12:51:05.047045 65034 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0924 12:51:05.047081 65034 iamrole.go:89] ListRoles: { } I0924 12:51:05.047127 65034 zones.go:93] ListHostedZonesPages { } I0924 12:51:05.047155 65034 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0924 12:51:05.047196 65034 internetgateways.go:101] DescribeInternetGateways: { } I0924 12:51:05.047239 65034 routetable.go:53] DescribeRouteTables: { } I0924 12:51:05.047273 65034 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0924 12:51:05.047707 65034 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0924 12:51:05.047714 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-group:nodes.minimal.example.com ok I0924 12:51:05.047817 65034 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0924 12:51:05.047827 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok autoscaling-config:lt-1 ok I0924 12:51:05.047898 65034 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:51:05.048011 65034 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0924 12:51:05.047905 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0924 12:51:05.047828 65034 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0924 12:51:05.048081 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0924 12:51:05.048243 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:05.048269 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0924 12:51:05.048302 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0924 12:51:05.048363 65034 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0924 12:51:05.048419 65034 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0924 12:51:05.048466 65034 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0924 12:51:05.048725 65034 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0924 12:51:05.048751 65034 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0924 12:51:05.048800 65034 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0924 12:51:05.048946 65034 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0924 12:51:05.048962 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0924 12:51:05.049050 65034 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0924 12:51:05.049079 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0924 12:51:05.049116 65034 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0924 12:51:05.049151 65034 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0924 12:51:05.049194 65034 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0924 12:51:05.049208 65034 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0924 12:51:05.049237 65034 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0924 12:51:05.049290 65034 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0924 12:51:05.049357 65034 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0924 12:51:05.049427 65034 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0924 12:51:05.049690 65034 featureflag.go:166] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleManyAddons (1.78s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 53.131s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.005s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0924 12:50:11.938611 65038 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.004s === RUN TestPodController I0924 12:50:11.966203 65041 pod.go:59] starting pod controller I0924 12:50:11.966431 65041 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) I0924 12:50:11.966500 65041 pod.go:107] Got stop signal PASS I0924 12:50:11.966526 65041 pod.go:65] shutting down pod controller ok k8s.io/kops/dns-controller/pkg/watchers 0.036s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.004s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035260} route53_test.go:93: Got zone list: [0xc000035290] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc0000940b8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000353c0} route53_test.go:93: Got zone list: [0xc0000353f0] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc0000940d0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000354f0} route53_test.go:170: Successfully added managed DNS zone: &{0xc000381e00 0xc0000940f0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000356b0} route53_test.go:93: Got zone list: [0xc0000356e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094110} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035840} route53_test.go:93: Got zone list: [0xc000035870] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094138} route53_test.go:187: Successfully added resource record set: {0xc0000fe600 0xc000094158} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035b50} route53_test.go:93: Got zone list: [0xc000035b80] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc0000941e8} route53_test.go:199: Successfully added resource record set: {0xc0000fe780 0xc000094208} route53_test.go:118: Got 1 recordsets: [0xc000035d20] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000035eb0} route53_test.go:93: Got zone list: [0xc000035ee0] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc0000942a8} route53_test.go:221: Successfully added resource record set: {0xc0000fe900 0xc0000942c8} route53_test.go:228: Correctly failed to add duplicate resource record {0xc0000fe900 0xc0000942c8}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ac2a0} route53_test.go:93: Got zone list: [0xc0003ac2d0] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094388} route53_test.go:246: Successfully removed resource set {0xc0000feb00 0xc0000943a8} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ac5b0} route53_test.go:93: Got zone list: [0xc0003ac5e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094438} route53_test.go:264: Successfully removed resource set {0xc0000fec80 0xc000094458} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ac8f0} route53_test.go:93: Got zone list: [0xc0003ac920] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc0000944f0} commontests.go:192: Successfully added resource record set: {0xc0000fee00 0xc000094510} commontests.go:45: Correctly replaced resource record {0xc0000fee00 0xc000094510} -> {0xc0000fef00 0xc000094560} commontests.go:142: Got 1 recordsets: [0xc0003acc30] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ace60} route53_test.go:93: Got zone list: [0xc0003ace90] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094618} commontests.go:192: Successfully added resource record set: {0xc0000ff180 0xc000094638} commontests.go:70: Correctly replaced resource record {0xc0000ff180 0xc000094638} -> {0xc0000ff280 0xc000094688} commontests.go:142: Got 1 recordsets: [0xc0003ad1a0] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ad420} route53_test.go:93: Got zone list: [0xc0003ad450] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094750} commontests.go:192: Successfully added resource record set: {0xc0000ff500 0xc000094770} commontests.go:142: Got 2 recordsets: [0xc0003ad6c0 0xc0003ad6e0] commontests.go:142: Got 2 recordsets: [0xc0003ad750 0xc0003ad770] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003ad970} route53_test.go:93: Got zone list: [0xc0003ad9a0] route53_test.go:98: Got at least 1 zone in list:&{0xc000381bc0 0xc000094878} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.014s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.007s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0002915e0} clouddns_test.go:71: Got zone list: [0xc000282a38] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000282a08} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0002916e0} clouddns_test.go:71: Got zone list: [0xc000282af8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000282ac8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0002917e0} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0002d0500 0xc000282bb8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0002918a0} clouddns_test.go:71: Got zone list: [0xc000282c78] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000282c48} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0002919c0} clouddns_test.go:71: Got zone list: [0xc000282db0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000282d80} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc000291b00} clouddns_test.go:71: Got zone list: [0xc000282f18] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000282ee8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc000291c60} clouddns_test.go:71: Got zone list: [0xc000283128] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc0002830f8} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc000291dc0} clouddns_test.go:71: Got zone list: [0xc0002832c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000283290} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc000291f00} clouddns_test.go:71: Got zone list: [0xc000283428] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc0002833f8} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0000cc160} clouddns_test.go:71: Got zone list: [0xc0002835f0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc0002835c0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0000cc4a0} clouddns_test.go:71: Got zone list: [0xc0002838c0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000283890} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0000cc620} clouddns_test.go:71: Got zone list: [0xc000283bf0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000283bc0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000591368 0xc0000cc800} clouddns_test.go:71: Got zone list: [0xc000283fc8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002d0380 0xc000283f98} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.053s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.018s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.065s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0924 12:50:15.143373 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.143526 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.143564 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.143605 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.143635 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.143686 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.143726 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.143744 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.143769 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.143803 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.151405 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0924 12:50:15.154940 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.155028 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.155052 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.155078 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.155101 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.155130 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.155155 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.155172 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.155192 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.155208 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.156186 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0924 12:50:15.160895 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.160989 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.161019 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.161050 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.161075 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.161107 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.161134 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.161151 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.161174 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.161190 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.162301 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.164142 355 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0924 12:50:15.164174 355 assetstore.go:111] bin/containerd I0924 12:50:15.164191 355 assetstore.go:111] bin/containerd-shim I0924 12:50:15.164206 355 assetstore.go:111] bin/containerd-shim-runc-v1 I0924 12:50:15.164219 355 assetstore.go:111] bin/containerd-shim-runc-v2 I0924 12:50:15.164232 355 assetstore.go:111] bin/containerd-stress I0924 12:50:15.164247 355 assetstore.go:111] bin/ctr I0924 12:50:15.164315 355 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0924 12:50:15.164338 355 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0924 12:50:15.167539 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.167607 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.167628 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.167652 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.167675 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.167713 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.167738 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.167754 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.167775 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.167788 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.168742 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.169977 355 containerd.go:60] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0924 12:50:15.170850 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.170929 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.170949 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.170970 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.170993 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.171018 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.171083 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.171100 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.171120 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.171135 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.172719 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.175231 355 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0924 12:50:15.176298 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.176435 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.176464 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.176513 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.176553 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.176604 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.176684 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.176719 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.176754 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.176779 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.178454 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.180870 355 assetstore.go:108] Matching assets for "^docker/": I0924 12:50:15.180902 355 assetstore.go:111] docker/containerd I0924 12:50:15.180918 355 assetstore.go:111] docker/containerd-shim I0924 12:50:15.180946 355 assetstore.go:111] docker/ctr I0924 12:50:15.180959 355 assetstore.go:111] docker/docker I0924 12:50:15.180969 355 assetstore.go:111] docker/docker-init I0924 12:50:15.180981 355 assetstore.go:111] docker/docker-proxy I0924 12:50:15.180991 355 assetstore.go:111] docker/dockerd I0924 12:50:15.181019 355 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0924 12:50:15.185746 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.185882 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.185913 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.185967 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.186017 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.186076 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.186150 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.186201 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.186255 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.186293 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.187830 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.190250 355 assetstore.go:108] Matching assets for "^docker/": I0924 12:50:15.190283 355 assetstore.go:111] docker/containerd I0924 12:50:15.190301 355 assetstore.go:111] docker/containerd-shim I0924 12:50:15.190317 355 assetstore.go:111] docker/ctr I0924 12:50:15.190340 355 assetstore.go:111] docker/docker I0924 12:50:15.190347 355 assetstore.go:111] docker/docker-init I0924 12:50:15.190355 355 assetstore.go:111] docker/docker-proxy I0924 12:50:15.190363 355 assetstore.go:111] docker/dockerd I0924 12:50:15.190370 355 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0924 12:50:15.194502 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.194640 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.194672 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.194721 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.194753 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.194790 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.194853 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.194878 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.194908 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.194929 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.196431 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.198928 355 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0924 12:50:15.199151 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.199259 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.199292 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.199329 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.199366 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.199414 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.199484 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.199517 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.199553 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.199577 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.201189 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:15.203621 355 assetstore.go:108] Matching assets for "^docker/": I0924 12:50:15.203652 355 assetstore.go:111] docker/docker I0924 12:50:15.203669 355 assetstore.go:111] docker/docker-containerd I0924 12:50:15.203686 355 assetstore.go:111] docker/docker-containerd-ctr I0924 12:50:15.203700 355 assetstore.go:111] docker/docker-containerd-shim I0924 12:50:15.203715 355 assetstore.go:111] docker/docker-init I0924 12:50:15.203728 355 assetstore.go:111] docker/docker-proxy I0924 12:50:15.203740 355 assetstore.go:111] docker/docker-runc I0924 12:50:15.203753 355 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0924 12:50:15.211014 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.211125 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.211152 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.211181 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.211203 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.211232 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.211268 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.211287 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.211316 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.211332 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.213673 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0924 12:50:15.215983 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.216081 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.216104 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.216139 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.216163 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.216200 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.216226 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.216243 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.216265 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.216300 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.218119 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0924 12:50:15.220242 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.220312 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.220340 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.220362 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.220384 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.220408 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.220429 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.220447 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.220465 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.220479 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.222137 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0924 12:50:15.228050 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.228131 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.228154 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.228182 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.228205 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.228230 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.228254 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.228272 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.228295 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.228311 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.229928 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestAuditConfigAPIServerBuilder I0924 12:50:15.239600 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.239713 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.239744 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.239783 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.239810 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.239843 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.239871 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.239889 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.239912 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.239931 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.242125 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestAuditConfigAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0924 12:50:15.248864 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.248948 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.248974 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.249001 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.249024 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.249053 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.249079 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.249096 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.249120 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.249137 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.250782 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0924 12:50:15.256872 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.256948 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.256972 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.256997 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.257021 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.257051 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.257075 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.257093 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.257115 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.257130 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.258701 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0924 12:50:15.266419 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.266514 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.266542 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.266573 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.266597 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.266627 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.266653 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.266668 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.266690 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.266706 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.268490 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0924 12:50:15.276261 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:15.276349 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:15.276376 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:15.276404 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:15.276426 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:15.276454 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:15.276479 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:15.276496 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:15.276520 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:15.276536 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:15.278290 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (0.79s) === RUN TestKubeAPIServerBuilderARM64 I0924 12:50:16.064335 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.064424 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.064450 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.064482 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.064505 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.064536 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.064560 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.064577 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.064620 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.064640 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.067767 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.13s) === RUN TestKubeControllerManagerBuilder I0924 12:50:16.195236 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.195327 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.195355 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.195389 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.195414 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.195446 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.195471 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.195490 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.195513 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.195530 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.197207 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0924 12:50:16.203276 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.203338 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.203359 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.203381 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.203405 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.203431 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.203451 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.203470 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.203490 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.203504 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.205095 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.13s) === RUN TestKubeControllerManagerBuilderARM64 I0924 12:50:16.334216 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.334306 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.334335 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.334370 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.334399 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.334430 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.334456 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.334473 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.334495 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.334511 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.336274 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.13s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0924 12:50:16.464410 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.464550 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.464611 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.464663 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.464704 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.464756 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.464799 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.464830 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.464868 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.464896 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.467615 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0924 12:50:16.473569 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.473680 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.473714 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.473752 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.473789 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.473832 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.473868 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.473897 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.473931 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.473955 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.476725 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.13s) === RUN TestKubeProxyBuilderARM64 I0924 12:50:16.599695 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.599781 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.599809 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.599838 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.599863 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.599892 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.599916 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.599935 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.599959 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.599975 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.601698 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.13s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0924 12:50:16.725654 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.725768 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.725802 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.725840 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.725878 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.725928 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.725964 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.726019 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.726060 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.726084 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.730935 355 zones.go:93] ListHostedZonesPages { } W0924 12:50:16.735367 355 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0924 12:50:16.738333 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.738455 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.738490 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.738533 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.738572 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.738621 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.738662 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.738693 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.738730 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.738759 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.741580 355 zones.go:93] ListHostedZonesPages { } W0924 12:50:16.863724 355 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.13s) === RUN TestKubeSchedulerBuilderARM64 I0924 12:50:16.866635 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.866746 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.866781 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.866828 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.866866 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.866912 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.866951 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.866979 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.867018 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.867044 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:16.869862 355 zones.go:93] ListHostedZonesPages { } W0924 12:50:16.993998 355 kube_scheduler.go:114] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.13s) === RUN TestKubectlBuilder I0924 12:50:16.996946 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:16.997059 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:16.997094 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:16.997136 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:16.997174 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:16.997220 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:16.997260 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:16.997291 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:16.997333 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:16.997360 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.000121 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:17.002885 355 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0924 12:50:17.005862 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:17.005948 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:17.006012 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:17.006047 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:17.006087 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:17.006130 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:17.006165 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:17.006196 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:17.006232 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:17.006256 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.008430 355 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0924 12:50:17.016016 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:17.016115 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:17.016174 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:17.016213 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:17.016251 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:17.016292 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:17.016327 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:17.016356 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:17.016391 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:17.016414 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.018449 355 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0924 12:50:17.023814 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:17.023928 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:17.023964 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:17.024003 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:17.024041 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:17.024083 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:17.024117 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:17.024146 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:17.024181 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:17.024208 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.026843 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:17.029600 355 assetstore.go:108] Matching assets for "protokube$": I0924 12:50:17.029633 355 assetstore.go:111] /opt/kops/bin/protokube I0924 12:50:17.029664 355 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0924 12:50:17.029705 355 assetstore.go:108] Matching assets for "channels$": I0924 12:50:17.029723 355 assetstore.go:111] /opt/kops/bin/channels I0924 12:50:17.029736 355 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0924 12:50:17.031899 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:17.032008 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:17.032042 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:17.032078 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:17.032114 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:17.032159 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:17.032198 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:17.032225 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:17.032259 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:17.032284 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.034877 355 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.01s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0924 12:50:17.038442 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:17.038553 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:17.038587 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:17.038627 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:17.038664 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:17.038705 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:17.039098 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:17.039126 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:17.039163 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:17.039188 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.041785 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:17.044404 355 update_service.go:91] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.01s) === RUN TestUpdateServiceBuilderExternal I0924 12:50:17.045022 355 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:17.045135 355 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:17.045169 355 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:17.045207 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:17.045247 355 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:17.045290 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:17.045325 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:17.045353 355 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:17.045389 355 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:17.045414 355 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:17.048033 355 zones.go:93] ListHostedZonesPages { } I0924 12:50:17.050663 355 update_service.go:86] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 1.974s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.020s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.024s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.019s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.020s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0924 12:50:15.101246 360 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0924 12:50:15.101390 360 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.016s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0924 12:50:17.220861 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.220986 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221035 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221105 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221147 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0924 12:50:17.221426 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221472 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221504 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221539 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221582 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221616 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221655 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221686 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221723 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221755 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221794 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221836 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221868 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221898 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221928 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.221974 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.222005 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.222034 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } I0924 12:50:17.222064 749 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0924 12:50:17.222266 749 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0924 12:50:17.222343 749 images.go:53] DescribeImagesPages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.081s === RUN TestManagedFieldsMigrator === RUN TestManagedFieldsMigrator/csa-migration === RUN TestManagedFieldsMigrator/merge-owners === RUN TestManagedFieldsMigrator/ssa-unchanged === RUN TestManagedFieldsMigrator/unknown-unchanged --- PASS: TestManagedFieldsMigrator (0.01s) --- PASS: TestManagedFieldsMigrator/csa-migration (0.00s) --- PASS: TestManagedFieldsMigrator/merge-owners (0.00s) --- PASS: TestManagedFieldsMigrator/ssa-unchanged (0.00s) --- PASS: TestManagedFieldsMigrator/unknown-unchanged (0.01s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.036s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0924 12:50:17.574093 751 builder.go:230] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.39s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0924 12:50:17.574965 751 builder.go:230] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0924 12:50:17.575782 751 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0924 12:50:17.576577 751 builder.go:230] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0924 12:50:17.577397 751 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0924 12:50:17.578223 751 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0924 12:50:17.578908 751 builder.go:230] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0924 12:50:17.579723 751 copyfile.go:208] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0924 12:50:17.579740 751 copyfile.go:209] S3, Google Cloud Storage, and File Paths are supported. E0924 12:50:17.579751 751 copyfile.go:210] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0924 12:50:17.579762 751 copyfile.go:211] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.427s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.004s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.073s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.041s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.092s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.024s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.005s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.003s === RUN TestHasExtraFields === RUN TestHasExtraFields/minimal === RUN TestHasExtraFields/extraFields === RUN TestHasExtraFields/spec2 === RUN TestHasExtraFields/isolateMasters === RUN TestHasExtraFields/instanceGroup --- PASS: TestHasExtraFields (0.01s) --- PASS: TestHasExtraFields/minimal (0.01s) --- PASS: TestHasExtraFields/extraFields (0.00s) --- PASS: TestHasExtraFields/spec2 (0.00s) --- PASS: TestHasExtraFields/isolateMasters (0.00s) --- PASS: TestHasExtraFields/instanceGroup (0.00s) PASS ok k8s.io/kops/pkg/edit 0.043s === RUN TestFlagToFalse I0924 12:50:23.551312 1776 featureflag_test.go:32] Created flag Unittest1 I0924 12:50:23.551381 1776 featureflag.go:166] FeatureFlag "UnitTest1"=false I0924 12:50:23.551401 1776 featureflag.go:166] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0924 12:50:23.551506 1776 featureflag.go:166] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) === RUN TestGetPositive --- PASS: TestGetPositive (0.00s) === RUN TestGetNegative --- PASS: TestGetNegative (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.005s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.032s === RUN TestRenderInstanceGroupZones W0924 12:50:23.569516 1778 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.023s === RUN TestWarmPoolOnlyRoll I0924 12:50:37.000640 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.000715 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.000733 2267 instancegroups.go:136] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0924 12:50:37.001123 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.001145 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.002306 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.002345 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.007510 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.007575 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.007598 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.007616 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.009745 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.010315 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.010409 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.010439 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.011573 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.011606 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.016870 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.016936 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.017524 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.017596 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.017651 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.019753 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.019797 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.024995 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.025049 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.025074 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.025094 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.026215 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.027006 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.027056 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.028197 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.028223 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.033422 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.033478 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.034392 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.034426 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.035549 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.035576 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.040785 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.040847 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.041493 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.041520 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.042634 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.042659 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.047872 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.047909 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.047940 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.047955 2267 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0924 12:50:37.048606 2267 instancegroups.go:431] Draining the node: "node-2a.local". I0924 12:50:37.049108 2267 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0924 12:50:37.049134 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.050266 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.050297 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.055523 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.055575 2267 instancegroups.go:431] Draining the node: "node-2b.local". I0924 12:50:37.056253 2267 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0924 12:50:37.056280 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.057404 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.057431 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.062596 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.062661 2267 instancegroups.go:431] Draining the node: "node-2c.local". I0924 12:50:37.063404 2267 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0924 12:50:37.063438 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.064574 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.064615 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.070817 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.070883 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0924 12:50:37.071468 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.071501 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.072648 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.072684 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.072706 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.072715 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.072734 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.073844 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.073880 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.073894 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.073919 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.075022 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.075047 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.075082 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.075095 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.075119 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.076234 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.076267 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.076281 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.076307 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.077413 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.077454 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.077468 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.077493 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.078607 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.078625 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.078649 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.078659 2267 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0924 12:50:37.078673 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.079794 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.079838 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.079852 2267 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0924 12:50:37.079874 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.081002 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. W0924 12:50:37.081046 2267 instancegroups.go:428] Not draining cluster nodes as 'cloudonly' flag is set. I0924 12:50:37.081062 2267 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0924 12:50:37.081088 2267 instancegroups.go:467] waiting for 1ms after terminating instance W0924 12:50:37.082214 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0924 12:50:37.082256 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0924 12:50:37.082605 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.082636 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.083732 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.083767 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.083782 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.083812 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.083825 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.083833 2267 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.083841 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.084369 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.084824 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.084877 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.084895 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.086017 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.086049 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.086070 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.086106 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.086902 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.086956 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.086986 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.088107 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.088128 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.088140 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.088155 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.088166 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.088174 2267 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.088183 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.088786 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.089446 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.089475 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.090608 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.090642 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.090663 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.090700 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.091551 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.091585 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.092711 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.092741 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.092760 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.092802 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.093577 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.093609 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.094739 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.094774 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.094794 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.094810 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.094830 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.094846 2267 instancegroups.go:513] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.094860 2267 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0924 12:50:37.095865 2267 instancegroups.go:431] Draining the node: "node-2a.local". I0924 12:50:37.096599 2267 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0924 12:50:37.096632 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.097765 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.097798 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.097819 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.097857 2267 instancegroups.go:431] Draining the node: "node-2b.local". I0924 12:50:37.098695 2267 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0924 12:50:37.098728 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.099861 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.099882 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.099894 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.099920 2267 instancegroups.go:431] Draining the node: "node-2c.local". I0924 12:50:37.100433 2267 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0924 12:50:37.100453 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.101591 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.101613 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. W0924 12:50:37.101626 2267 instancegroups.go:513] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0924 12:50:37.101640 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0924 12:50:37.101910 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0924 12:50:37.102117 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.102135 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.103252 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.103272 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.108481 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.108550 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.108566 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.108576 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.109189 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.109642 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.109701 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.109726 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.110861 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.110893 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.116091 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.116166 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.116796 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.116856 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.116878 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.118030 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.118053 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.123244 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.123283 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.123296 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.123306 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.123959 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.124494 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.124518 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.125668 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.125689 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.130895 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.130962 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.131611 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.131638 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.132777 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.132860 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.138129 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.138198 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.138845 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.138872 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.139976 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.140000 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.145214 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.145262 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.145286 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.145303 2267 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0924 12:50:37.146414 2267 instancegroups.go:431] Draining the node: "node-2a.local". I0924 12:50:37.147232 2267 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0924 12:50:37.147271 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.148452 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.148488 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.153736 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.153816 2267 instancegroups.go:431] Draining the node: "node-2b.local". I0924 12:50:37.154842 2267 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0924 12:50:37.154881 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.155983 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.156014 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.161224 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.161315 2267 instancegroups.go:431] Draining the node: "node-2c.local". I0924 12:50:37.162234 2267 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0924 12:50:37.162273 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.163404 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.163433 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.168661 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.168713 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0924 12:50:37.168907 2267 rollingupdate.go:111] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0924 12:50:37.169325 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.169354 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.170515 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.170542 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.170555 2267 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0924 12:50:37.170811 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.170829 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.171958 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.171985 2267 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0924 12:50:37.172005 2267 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0924 12:50:37.172238 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.172265 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.172283 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.172300 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0924 12:50:37.172474 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.172497 2267 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0924 12:50:37.172515 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.172532 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0924 12:50:37.172752 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.172781 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.173922 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.173945 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.179195 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.179279 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.179310 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.179329 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.180175 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.180929 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.181001 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.181036 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.182171 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.182214 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.182235 2267 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0924 12:50:37.182601 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.182629 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.183771 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.183804 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.189054 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.189133 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.189159 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.189169 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.189727 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.190198 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.190252 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.190272 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.191458 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.191510 2267 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0924 12:50:37.191538 2267 instancegroups.go:506] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0924 12:50:37.191908 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.191938 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.191955 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.193100 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.193935 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.193972 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.195113 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.195150 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.200396 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.200476 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.201314 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.201341 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.202512 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.202539 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.207787 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.207868 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.208716 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.208751 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.209951 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.209987 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.215238 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.215292 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0924 12:50:37.215665 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.215700 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.215724 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.215741 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0924 12:50:37.216007 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.216033 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.216052 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.216068 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0924 12:50:37.216306 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.216331 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.216347 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.216364 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupExitableError I0924 12:50:37.216632 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.216655 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.216668 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.217746 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.220859 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.220909 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.222051 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.222096 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.227237 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.227311 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.228016 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.228042 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.229170 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.229199 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.234425 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.234503 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.235390 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.235424 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.236608 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.236631 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.241837 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.241888 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.241920 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.241948 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.241967 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-2": error validating cluster: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateValidationErrorInstanceGroupExitableError (0.03s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0924 12:50:37.242282 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.242307 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.242322 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.243378 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.243982 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.244008 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.245136 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.245180 2267 instancegroups.go:557] Cluster did not pass validation within deadline: testing failure. E0924 12:50:37.245201 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.245222 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0924 12:50:37.245438 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.245465 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.245479 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.246497 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.247260 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.247297 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.248420 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.248457 2267 instancegroups.go:547] Cluster did not validate within deadline: testing validation error. E0924 12:50:37.248483 2267 instancegroups.go:506] Cluster did not validate within 0s E0924 12:50:37.248504 2267 rollingupdate.go:218] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0924 12:50:37.248752 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.248776 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.249914 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.249967 2267 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0924 12:50:37.251118 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.256381 2267 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0924 12:50:37.257576 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.266432 2267 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0924 12:50:37.268262 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.273522 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.273626 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.273657 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.273673 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.274545 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.275315 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.275391 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.275423 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.278813 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.278888 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.284133 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.284207 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.284914 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.284978 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.284999 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.286124 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.286177 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.291388 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.291432 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.291458 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.291473 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.292382 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.293103 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.293132 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.294274 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.294305 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.299513 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.299569 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.300367 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.300397 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.301548 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.301593 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.306834 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.306918 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.307858 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.307895 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.309039 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.309089 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.314345 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.314384 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.314414 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.314431 2267 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0924 12:50:37.315390 2267 instancegroups.go:431] Draining the node: "node-2a.local". I0924 12:50:37.316081 2267 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0924 12:50:37.316113 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.317272 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.317313 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.322554 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.322695 2267 instancegroups.go:431] Draining the node: "node-2b.local". I0924 12:50:37.323594 2267 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0924 12:50:37.323662 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.324847 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.324917 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.330173 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.330240 2267 instancegroups.go:431] Draining the node: "node-2c.local". I0924 12:50:37.330860 2267 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0924 12:50:37.330885 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.332028 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.332093 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.337309 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.337362 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I0924 12:50:37.337611 2267 instancegroups.go:627] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0924 12:50:37.337634 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.338767 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.338812 2267 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0924 12:50:37.339944 2267 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0924 12:50:37.341086 2267 instancegroups.go:560] Cluster did not pass validation, will retry in "1ms": testing failure. I0924 12:50:37.342210 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.347424 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.347521 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.347550 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.347569 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.348481 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.349262 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.349335 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.349370 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.350547 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.350585 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.355794 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.355867 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.356771 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.356841 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.356876 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.358048 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.358089 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.363313 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.363363 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.363378 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.363403 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.364385 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.364931 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.364956 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.366077 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.366104 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.371309 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.371371 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.371985 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.372009 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.373123 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.373148 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.378358 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.378416 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.379012 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.379036 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.380147 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.380173 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.385385 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.385420 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.385434 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.385445 2267 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0924 12:50:37.386108 2267 instancegroups.go:431] Draining the node: "node-2a.local". I0924 12:50:37.386589 2267 instancegroups.go:625] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0924 12:50:37.386613 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.387727 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.387755 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.392958 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.393019 2267 instancegroups.go:431] Draining the node: "node-2b.local". I0924 12:50:37.393612 2267 instancegroups.go:625] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0924 12:50:37.393637 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.394753 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.394780 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.399987 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.400041 2267 instancegroups.go:431] Draining the node: "node-2c.local". I0924 12:50:37.400637 2267 instancegroups.go:625] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0924 12:50:37.400660 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.401780 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.401806 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.407022 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.407064 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0924 12:50:37.407567 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.407584 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.407592 2267 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0924 12:50:37.408104 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.408575 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.408599 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.409739 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.409779 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.415023 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.415084 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.415903 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.415930 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.417076 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.417097 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.422322 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.422362 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.01s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0924 12:50:37.422612 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.422630 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.422638 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.423182 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.423604 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.423656 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.423674 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.424818 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.424838 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.430029 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.430078 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.430724 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.430783 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.430818 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.431955 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.431978 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.437197 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.437247 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.01s) === RUN TestRollingUpdateDisabled I0924 12:50:37.437617 2267 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled W0924 12:50:37.437657 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0924 12:50:37.437681 2267 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled W0924 12:50:37.437698 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0924 12:50:37.437723 2267 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled W0924 12:50:37.437737 2267 instancegroups.go:499] Not validating cluster as cloudonly flag is set. I0924 12:50:37.437756 2267 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0924 12:50:37.437774 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0924 12:50:37.438008 2267 instancegroups.go:604] Detaching instance "bastion-1a", in group "bastion-1". I0924 12:50:37.438062 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:37.439168 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.439200 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.444479 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.444539 2267 instancegroups.go:216] Rolling updates for InstanceGroup bastion-1 are disabled I0924 12:50:37.444626 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.444651 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.444668 2267 instancegroups.go:342] Tainting 2 nodes in "master-1" instancegroup. I0924 12:50:37.445459 2267 instancegroups.go:216] Rolling updates for InstanceGroup master-1 are disabled I0924 12:50:37.445481 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.445502 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.445515 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:37.446361 2267 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0924 12:50:37.446427 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:37.447546 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.447578 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.452789 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.452850 2267 instancegroups.go:216] Rolling updates for InstanceGroup node-1 are disabled I0924 12:50:37.452870 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.452895 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.452912 2267 instancegroups.go:342] Tainting 3 nodes in "node-2" instancegroup. I0924 12:50:37.454003 2267 instancegroups.go:602] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0924 12:50:37.454068 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:37.455182 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.455201 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:37.460419 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.460454 2267 instancegroups.go:216] Rolling updates for InstanceGroup node-2 are disabled I0924 12:50:37.460471 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0924 12:50:37.460703 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.460724 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.460734 2267 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0924 12:50:37.461992 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.462438 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.462466 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.463612 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.463639 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.463662 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.463699 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.464486 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.464688 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.464713 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.465818 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.465855 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.566239 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.566322 2267 instancegroups.go:431] Draining the node: "node-1e.local". I0924 12:50:37.566349 2267 instancegroups.go:431] Draining the node: "node-1d.local". I0924 12:50:37.567342 2267 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0924 12:50:37.567586 2267 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0924 12:50:37.567632 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.568763 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.568798 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.568827 2267 instancegroups.go:431] Draining the node: "node-1f.local". I0924 12:50:37.569555 2267 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0924 12:50:37.569590 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.570718 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.570765 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.671197 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.671287 2267 instancegroups.go:431] Draining the node: "node-1g.local". I0924 12:50:37.672267 2267 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0924 12:50:37.672316 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.673453 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.673490 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.673512 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0924 12:50:37.673767 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.673796 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.673809 2267 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0924 12:50:37.675491 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:37.675530 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:37.676395 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:37.676545 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:37.676571 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.677716 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.677778 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.778247 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.778309 2267 instancegroups.go:431] Draining the node: "node-1d.local". I0924 12:50:37.778339 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:37.779304 2267 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0924 12:50:37.779558 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:37.779601 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.780761 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.780790 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.780812 2267 instancegroups.go:431] Draining the node: "node-1e.local". I0924 12:50:37.781323 2267 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0924 12:50:37.781349 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.782466 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.782512 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.882956 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.883020 2267 instancegroups.go:431] Draining the node: "node-1f.local". I0924 12:50:37.883695 2267 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0924 12:50:37.883730 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.884852 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.884889 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.884908 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0924 12:50:37.885097 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.885121 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.885134 2267 instancegroups.go:342] Tainting 7 nodes in "master-1" instancegroup. I0924 12:50:37.886334 2267 instancegroups.go:431] Draining the node: "master-1a.local". I0924 12:50:37.886806 2267 instancegroups.go:688] skipping deregistration of instance "master-1a", as part of instancegroup with role "ControlPlane" I0924 12:50:37.886852 2267 instancegroups.go:625] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0924 12:50:37.886877 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.887958 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.887986 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.888008 2267 instancegroups.go:431] Draining the node: "master-1c.local". I0924 12:50:37.888043 2267 instancegroups.go:431] Draining the node: "master-1b.local". I0924 12:50:37.888815 2267 instancegroups.go:688] skipping deregistration of instance "master-1c", as part of instancegroup with role "ControlPlane" I0924 12:50:37.888852 2267 instancegroups.go:625] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0924 12:50:37.888986 2267 instancegroups.go:688] skipping deregistration of instance "master-1b", as part of instancegroup with role "ControlPlane" I0924 12:50:37.889018 2267 instancegroups.go:625] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0924 12:50:37.889040 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.890184 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.890235 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.990620 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.990698 2267 instancegroups.go:431] Draining the node: "master-1e.local". I0924 12:50:37.990736 2267 instancegroups.go:431] Draining the node: "master-1d.local". I0924 12:50:37.991627 2267 instancegroups.go:688] skipping deregistration of instance "master-1d", as part of instancegroup with role "ControlPlane" I0924 12:50:37.991686 2267 instancegroups.go:625] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0924 12:50:37.991930 2267 instancegroups.go:688] skipping deregistration of instance "master-1e", as part of instancegroup with role "ControlPlane" I0924 12:50:37.991983 2267 instancegroups.go:625] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0924 12:50:37.992023 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.993163 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.993201 2267 instancegroups.go:537] Cluster validated. I0924 12:50:37.993238 2267 instancegroups.go:431] Draining the node: "master-1f.local". I0924 12:50:37.994018 2267 instancegroups.go:688] skipping deregistration of instance "master-1f", as part of instancegroup with role "ControlPlane" I0924 12:50:37.994076 2267 instancegroups.go:625] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0924 12:50:37.994114 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:37.995245 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:37.995299 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.095734 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.095810 2267 instancegroups.go:431] Draining the node: "master-1g.local". I0924 12:50:38.096420 2267 instancegroups.go:688] skipping deregistration of instance "master-1g", as part of instancegroup with role "ControlPlane" I0924 12:50:38.096476 2267 instancegroups.go:625] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0924 12:50:38.096506 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.097648 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.097679 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.097691 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.21s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0924 12:50:38.097876 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.097897 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.097905 2267 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0924 12:50:38.098922 2267 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0924 12:50:38.098965 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.100111 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.100140 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.100149 2267 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0924 12:50:38.100183 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.101293 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.101320 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.101355 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.101384 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.103887 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.103953 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.104019 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.105200 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.105252 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.205621 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.205697 2267 instancegroups.go:431] Draining the node: "node-1d.local". I0924 12:50:38.205770 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:38.206349 2267 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0924 12:50:38.206820 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:38.206859 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.207995 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.208026 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.208057 2267 instancegroups.go:431] Draining the node: "node-1e.local". I0924 12:50:38.208512 2267 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0924 12:50:38.208545 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.209664 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.209712 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.310152 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.310215 2267 instancegroups.go:431] Draining the node: "node-1f.local". I0924 12:50:38.310840 2267 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0924 12:50:38.310879 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.312003 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.312041 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.312061 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0924 12:50:38.312290 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.312319 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.312333 2267 instancegroups.go:342] Tainting 6 nodes in "node-1" instancegroup. I0924 12:50:38.313923 2267 instancegroups.go:602] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0924 12:50:38.313969 2267 instancegroups.go:602] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0924 12:50:38.314015 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.315172 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.315202 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.315232 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.315255 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.315781 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.316142 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.316169 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.317284 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.317324 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.417702 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.417769 2267 instancegroups.go:431] Draining the node: "node-1d.local". I0924 12:50:38.417833 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:38.418463 2267 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0924 12:50:38.419090 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:38.419133 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.420263 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.420298 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.420329 2267 instancegroups.go:431] Draining the node: "node-1e.local". I0924 12:50:38.421047 2267 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0924 12:50:38.421090 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.422211 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.422259 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.524678 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.524740 2267 instancegroups.go:431] Draining the node: "node-1f.local". I0924 12:50:38.525366 2267 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0924 12:50:38.525407 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.526552 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.526603 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.526624 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0924 12:50:38.526861 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.526883 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.526894 2267 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0924 12:50:38.527555 2267 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0924 12:50:38.527582 2267 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0924 12:50:38.527615 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.528767 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.528791 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.534008 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.534091 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.534099 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.535244 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.535285 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.535521 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.535561 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.536457 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.536501 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.541711 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.541763 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0924 12:50:38.542059 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.542086 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.542098 2267 instancegroups.go:342] Tainting 2 nodes in "node-1" instancegroup. I0924 12:50:38.542797 2267 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0924 12:50:38.542853 2267 instancegroups.go:191] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0924 12:50:38.542866 2267 instancegroups.go:602] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0924 12:50:38.542903 2267 instancegroups.go:191] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0924 12:50:38.542933 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.544098 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.544131 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.549333 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.549395 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.549412 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.550480 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.550513 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.550703 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.550732 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.551674 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.551698 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.556934 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.556980 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0924 12:50:38.557257 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.557292 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.557305 2267 instancegroups.go:342] Tainting 4 nodes in "node-1" instancegroup. I0924 12:50:38.558548 2267 instancegroups.go:602] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0924 12:50:38.558608 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.559777 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.559815 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.559824 2267 instancegroups.go:602] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0924 12:50:38.559865 2267 instancegroups.go:203] waiting for 1ms after detaching instance I0924 12:50:38.560981 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.561009 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.561045 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:38.561064 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.561089 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.562343 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:38.562389 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.562570 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.562608 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.562832 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.562871 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.563546 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.664434 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.664510 2267 instancegroups.go:431] Draining the node: "node-1d.local". I0924 12:50:38.665510 2267 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0924 12:50:38.665562 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.666693 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.666728 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.666747 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0924 12:50:38.666976 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.667003 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.667015 2267 instancegroups.go:342] Tainting 7 nodes in "node-1" instancegroup. I0924 12:50:38.668893 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.669557 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.669593 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.670709 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.670739 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.670772 2267 instancegroups.go:431] Draining the node: "node-1e.local". I0924 12:50:38.670832 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:38.671735 2267 instancegroups.go:625] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0924 12:50:38.671986 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:38.672020 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.673137 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.673179 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.773812 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.773882 2267 instancegroups.go:431] Draining the node: "node-1g.local". I0924 12:50:38.773911 2267 instancegroups.go:431] Draining the node: "node-1f.local". I0924 12:50:38.774921 2267 instancegroups.go:625] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0924 12:50:38.775174 2267 instancegroups.go:625] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0924 12:50:38.775215 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.776369 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.776399 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.776420 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.776911 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.776936 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.778051 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.778093 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.878677 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.878753 2267 instancegroups.go:431] Draining the node: "node-1d.local". I0924 12:50:38.879745 2267 instancegroups.go:625] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0924 12:50:38.879797 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.880961 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.880992 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.881007 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0924 12:50:38.881187 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.881203 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.881212 2267 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0924 12:50:38.881231 2267 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0924 12:50:38.881241 2267 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0924 12:50:38.881268 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0924 12:50:38.881374 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.881389 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.881396 2267 instancegroups.go:342] Tainting 3 nodes in "node-1" instancegroup. I0924 12:50:38.881970 2267 instancegroups.go:136] deleting warm pool instance "node-1-wp-a" I0924 12:50:38.881988 2267 instancegroups.go:136] deleting warm pool instance "node-1-wp-b" I0924 12:50:38.881997 2267 instancegroups.go:136] deleting warm pool instance "node-1-wp-c" I0924 12:50:38.882019 2267 instancegroups.go:431] Draining the node: "node-1a.local". I0924 12:50:38.882460 2267 instancegroups.go:625] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0924 12:50:38.882480 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.883601 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.883627 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.888831 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.888874 2267 instancegroups.go:431] Draining the node: "node-1b.local". I0924 12:50:38.889471 2267 instancegroups.go:625] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0924 12:50:38.889494 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.890988 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.891019 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.897883 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.897975 2267 instancegroups.go:431] Draining the node: "node-1c.local". I0924 12:50:38.898890 2267 instancegroups.go:625] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0924 12:50:38.898924 2267 instancegroups.go:467] waiting for 1ms after terminating instance I0924 12:50:38.900081 2267 instancegroups.go:501] Validating the cluster. I0924 12:50:38.900104 2267 instancegroups.go:540] Cluster validated; revalidating in 5ms to make sure it does not flap. I0924 12:50:38.905280 2267 instancegroups.go:537] Cluster validated. I0924 12:50:38.905304 2267 rollingupdate.go:234] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.999s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.005s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.014s === RUN TestParse E0924 12:50:36.922544 2270 versions.go:96] unable to parse Kubernetes version "" E0924 12:50:36.922679 2270 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.012s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.034s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option === RUN TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option --- PASS: TestBuildKubecfg (0.16s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_with_admin_and_internal_option (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_for_Public_cluster_without_admin_and_with_internal_option (0.00s) PASS ok k8s.io/kops/pkg/kubeconfig 0.227s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.097s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/ControlPlane (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.085s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.084s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0924 12:50:36.977382 2276 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.071s === RUN Test_RunEtcdManagerBuilder I0924 12:50:36.995700 2277 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0924 12:50:37.036124 2277 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 W0924 12:50:37.037543 2277 model.go:339] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.05s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.134s === RUN Test_RunKubeApiserverBuilder I0924 12:50:36.985861 2278 featureflag.go:166] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.095s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.095s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.028s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.058s === RUN TestServerGroupBuilder I0924 12:50:44.622691 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.636807 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.652845 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.673028 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.718009 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.735736 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.754787 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.770782 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.786974 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.856429 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.867149 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.873408 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.879456 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.885188 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.892867 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.898580 3103 featureflag.go:166] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.28s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.368s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.027s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.012s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.026s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (0.86s) --- PASS: TestIssueCert/ca (0.44s) --- PASS: TestIssueCert/client (0.25s) --- PASS: TestIssueCert/clientOneYear (0.17s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 0.880s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.004s === RUN TestAddUntaggedRouteTables I0924 12:50:44.607567 3110 routetable.go:53] DescribeRouteTables: { } I0924 12:50:44.607707 3110 aws.go:229] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W0924 12:50:44.607730 3110 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0924 12:50:44.607887 3110 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0924 12:50:44.608001 3110 aws.go:2059] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0924 12:50:44.608120 3110 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0924 12:50:44.608280 3110 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0924 12:50:44.608379 3110 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0924 12:50:44.608582 3110 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0924 12:50:44.608714 3110 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0924 12:50:44.608763 3110 volumes.go:128] DescribeVolumes: { } W0924 12:50:44.608820 3110 aws.go:2212] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0924 12:50:44.609030 3110 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:44.609152 3110 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:44.609203 3110 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:44.609279 3110 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:44.609320 3110 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:44.609407 3110 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:44.609472 3110 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:44.609507 3110 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:44.609559 3110 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:44.609592 3110 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:44.609639 3110 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0924 12:50:44.609681 3110 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0924 12:50:44.609717 3110 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0924 12:50:44.609771 3110 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0924 12:50:44.609808 3110 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0924 12:50:44.609856 3110 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0924 12:50:44.609926 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0924 12:50:44.609963 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0924 12:50:44.609999 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0924 12:50:44.610032 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0924 12:50:44.610069 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0924 12:50:44.610105 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0924 12:50:44.610141 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0924 12:50:44.610173 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0924 12:50:44.610209 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0924 12:50:44.610243 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0924 12:50:44.610282 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0924 12:50:44.610325 3110 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.070s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.038s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.068s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.005s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.004s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.003s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.005s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.005s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.039s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.105s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.005s === RUN TestRemovesDuplicateGuardedBlocks W0924 12:50:51.986269 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986323 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986334 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986347 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986356 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986365 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986374 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986381 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986388 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986395 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986402 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986410 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986424 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986433 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.986440 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.986447 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.988872 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.988914 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989017 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989040 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989129 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989148 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989238 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989256 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989345 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989364 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989468 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989489 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989585 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989603 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989704 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989723 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989818 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989836 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.989954 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.989974 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990071 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990090 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990175 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990193 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990278 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990297 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990382 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990400 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990487 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990504 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990594 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990612 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990697 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990714 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990799 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990818 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.990906 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.990924 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991067 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991086 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991177 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991196 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991306 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991326 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991413 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991430 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991517 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991535 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991618 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991652 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991740 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991757 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991844 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991869 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.991954 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.991970 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992058 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992074 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992166 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992184 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992271 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992288 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992378 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992394 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992481 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992498 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992583 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992599 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992686 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992703 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992800 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992818 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.992910 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.992928 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993012 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993028 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993143 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993160 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993258 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993274 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993407 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993424 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993513 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993529 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993629 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993645 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993730 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993747 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993828 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993844 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.993928 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.993943 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994027 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994059 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994162 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994178 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994262 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994277 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994360 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994377 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994462 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994479 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994563 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994580 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994686 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994702 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994786 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994800 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994885 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.994902 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.994990 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995006 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995092 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995108 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995194 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995211 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995294 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995314 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995398 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995418 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995506 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995525 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995612 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995631 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995745 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995763 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995854 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995873 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.995956 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.995974 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996060 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996078 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996214 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996232 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996317 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996335 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996424 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996442 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996530 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996549 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996635 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996654 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996741 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996758 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996843 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996860 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.996951 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.996969 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997053 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997071 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997156 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997175 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997259 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997277 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997361 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997378 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997469 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997488 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997576 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997594 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997679 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997696 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997783 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997801 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.997885 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.997903 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.998053 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.998072 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.998157 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.998175 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.998259 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.998276 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.998361 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.998379 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.998466 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.998483 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.998567 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.998585 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.999389 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.999413 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.999501 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.999565 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.999686 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.999738 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:51.999856 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:51.999905 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.000026 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.000077 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.000200 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.000251 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.000348 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.000364 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.000448 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.000463 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.000544 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.000561 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.000644 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.000660 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0924 12:50:52.001149 4405 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0924 12:50:52.001166 4405 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0924 12:50:52.001179 4405 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0924 12:50:52.001191 4405 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0924 12:50:52.001203 4405 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0924 12:50:52.001219 4405 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0924 12:50:52.001230 4405 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0924 12:50:52.001274 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001292 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001305 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.001318 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001330 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.001341 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001351 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.001363 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001376 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.001388 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001404 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001416 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001427 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001439 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.001451 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.001463 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.021890 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.021930 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022031 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022063 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022147 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022163 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022245 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022261 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022369 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022386 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022462 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022483 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022566 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022585 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022666 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022684 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022765 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022782 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022864 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022881 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.022974 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.022994 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023088 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023107 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023214 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023233 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023332 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023350 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023437 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023469 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023558 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023576 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023664 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023682 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023768 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023787 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023876 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.023898 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.023990 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024009 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024099 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024124 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024210 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024231 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024315 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024333 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024419 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024438 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024526 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024544 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024661 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024681 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024768 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024787 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.024880 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.024899 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025018 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025037 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025123 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025143 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025230 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025248 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025334 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025353 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025438 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025456 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025547 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025565 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025650 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025669 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025756 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025774 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025859 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025878 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.025967 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.025994 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026083 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026101 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026184 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026202 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026291 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026309 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026410 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026430 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026514 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026533 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026619 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026637 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026723 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026741 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026826 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026844 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.026930 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.026949 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027035 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027053 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027142 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027161 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027246 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027265 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027352 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027370 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027458 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027476 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027584 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027602 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027691 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027710 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027798 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027817 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.027905 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.027937 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028036 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028055 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028148 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028167 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028258 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028278 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028378 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028397 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028487 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028506 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028596 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028616 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028712 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028731 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028828 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028848 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.028944 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.028964 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029054 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029074 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029165 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029184 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029281 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029299 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029389 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029409 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029498 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029520 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029609 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029629 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029720 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029741 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029832 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029851 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.029942 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.029962 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030051 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030070 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030188 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030208 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030339 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030360 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030451 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030470 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030561 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030584 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030675 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030695 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030788 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030808 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.030899 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.030919 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031013 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031033 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031130 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031149 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031243 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031273 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031366 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031385 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031472 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031490 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031578 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031601 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031696 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031715 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031813 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031851 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.031942 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.031959 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032055 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032075 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032166 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032186 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032278 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032298 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032391 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032411 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032500 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032519 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032608 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032628 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032716 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032735 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0924 12:50:52.032824 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0924 12:50:52.032844 4405 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.03s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.051s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.032s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.00s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.01s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.068s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.035s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0924 12:50:52.008899 4409 files.go:57] Writing file "/tmp/TestWriteFile501085006/001/SubDir/test1.tmp" I0924 12:50:52.009043 4409 files.go:113] Changing file mode for "/tmp/TestWriteFile501085006/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.01s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.036s === RUN TestBootstrapChannelBuilder_BuildTasks I0924 12:50:54.718342 5097 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:54.718499 5097 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:54.718538 5097 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:54.718573 5097 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:54.718606 5097 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:54.718660 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:54.718694 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:54.718712 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:54.718748 5097 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:54.718775 5097 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:54.726585 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:54.772517 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:54.883177 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:54.952891 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:55.013040 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:55.071230 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:55.124101 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:55.200606 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:55.349877 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:55.927878 5097 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.26s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0924 12:50:55.975902 5097 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:55.975979 5097 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:55.976024 5097 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:55.976058 5097 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:55.976082 5097 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:55.976113 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:55.976140 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:55.976157 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:55.976180 5097 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:55.976198 5097 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:55.976215 5097 featureflag.go:169] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0924 12:50:55.977396 5097 zones.go:93] ListHostedZonesPages { } I0924 12:50:56.083352 5097 featureflag.go:169] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.11s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0924 12:50:56.083501 5097 featureflag.go:166] FeatureFlag "ImageDigest"=false I0924 12:50:56.083573 5097 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:56.083623 5097 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { VpcId: "vpc-12345678" } I0924 12:50:56.083657 5097 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0924 12:50:56.083683 5097 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0924 12:50:56.083716 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0924 12:50:56.083743 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0924 12:50:56.083760 5097 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0924 12:50:56.083783 5097 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0924 12:50:56.083800 5097 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0924 12:50:56.084999 5097 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.12s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.19s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.41s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0924 12:50:56.820462 5097 new_cluster.go:1132] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestDefaultImage --- PASS: TestDefaultImage (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0924 12:50:56.857213 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857258 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857280 5097 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0924 12:50:56.857316 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0924 12:50:56.857337 5097 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0924 12:50:56.857374 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0924 12:50:56.857410 5097 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0924 12:50:56.857443 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857466 5097 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0924 12:50:56.857502 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857525 5097 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0924 12:50:56.857561 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857615 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857656 5097 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0924 12:50:56.857677 5097 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0924 12:50:56.857698 5097 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0924 12:50:56.857719 5097 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0924 12:50:56.857739 5097 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0924 12:50:56.857760 5097 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0924 12:50:56.857782 5097 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0924 12:50:56.857804 5097 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0924 12:50:56.859638 5097 images.go:53] DescribeImagesPages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN TestKopsFeatureEnabled === RUN TestKopsFeatureEnabled/Missing_feature === RUN TestKopsFeatureEnabled/Existing_feature I0924 12:50:56.859930 5097 featureflag.go:166] FeatureFlag "Scaleway"=true --- PASS: TestKopsFeatureEnabled (0.00s) --- PASS: TestKopsFeatureEnabled/Missing_feature (0.00s) --- PASS: TestKopsFeatureEnabled/Existing_feature (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 2.236s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0924 12:50:54.703708 5096 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1542635427/001 I0924 12:50:54.704093 5096 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1542635427/001 I0924 12:50:54.704326 5096 target.go:116] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender1542635427/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0924 12:50:54.708723 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0924 12:50:54.708803 5096 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0924 12:50:54.708872 5096 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0924 12:50:54.708974 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.709121 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.709194 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.709214 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.709370 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.709404 5096 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0924 12:50:54.709437 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.709491 5096 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0924 12:50:54.709570 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0924 12:50:54.709625 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.709681 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.709720 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.709735 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.709768 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.709804 5096 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0924 12:50:54.709857 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0924 12:50:54.709978 5096 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0924 12:50:54.710024 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.710129 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0924 12:50:54.710169 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.710192 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.710216 5096 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0924 12:50:54.710331 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.710484 5096 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0924 12:50:54.710527 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.710548 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.710574 5096 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0924 12:50:54.710625 5096 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.710755 5096 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0924 12:50:54.710824 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.710847 5096 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" },{ Key: "AssociatedElasticIp", Value: "192.0.2.1" }] } I0924 12:50:54.710882 5096 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0924 12:50:54.710941 5096 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0924 12:50:54.710979 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.711021 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.711036 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.711060 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.711083 5096 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0924 12:50:54.711118 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.711170 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.711195 5096 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0924 12:50:54.711230 5096 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.711294 5096 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0924 12:50:54.711331 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0924 12:50:54.711368 5096 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0924 12:50:54.711423 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0924 12:50:54.711462 5096 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0924 12:50:54.711483 5096 internetgateways.go:66] CreateInternetGateway: { } I0924 12:50:54.711499 5096 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0924 12:50:54.711547 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.711582 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.711616 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.711632 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.711692 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.711717 5096 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0924 12:50:54.711745 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.711796 5096 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0924 12:50:54.711864 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0924 12:50:54.711913 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.711948 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.711983 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.712000 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.712050 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.712107 5096 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0924 12:50:54.712164 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0924 12:50:54.712581 5096 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1929449387/001 I0924 12:50:54.712797 5096 target.go:116] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1929449387/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0924 12:50:54.713142 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.713195 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.713276 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0924 12:50:54.713312 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.713335 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.713359 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.713433 5096 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0924 12:50:54.713610 5096 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0924 12:50:54.713650 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0924 12:50:54.713699 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0924 12:50:54.713756 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.713795 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.713834 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.713849 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.713876 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.713900 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.713938 5096 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0924 12:50:54.714006 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0924 12:50:54.714213 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.714251 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.714326 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0924 12:50:54.714360 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.714383 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.714406 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.714441 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.714521 5096 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0924 12:50:54.714560 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.714591 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.714623 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0924 12:50:54.714684 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.714722 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.714760 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.714775 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.714801 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.714823 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.714857 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.714906 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.714929 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0924 12:50:54.715026 5096 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0924 12:50:54.715063 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.715135 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0924 12:50:54.715170 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.715190 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.715213 5096 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0924 12:50:54.715252 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.715345 5096 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0924 12:50:54.715390 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.715413 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.715432 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.715586 5096 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:54.715645 5096 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0924 12:50:54.715739 5096 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0924 12:50:54.715788 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.715833 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.715848 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.715871 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.715895 5096 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0924 12:50:54.715931 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.715979 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.716033 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.716057 5096 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0924 12:50:54.716155 5096 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0924 12:50:54.716194 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.716266 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0924 12:50:54.716303 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.716324 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.716348 5096 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0924 12:50:54.716380 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.716456 5096 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0924 12:50:54.716491 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.716675 5096 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0924 12:50:54.716752 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.716782 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.716808 5096 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0924 12:50:54.716885 5096 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0924 12:50:54.716944 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.716992 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.717008 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.717033 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.717056 5096 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0924 12:50:54.717110 5096 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0924 12:50:54.717208 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.717249 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.717350 5096 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0924 12:50:54.717435 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0924 12:50:54.717531 5096 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0924 12:50:54.717638 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.717694 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.717752 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.717776 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.717871 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.717911 5096 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0924 12:50:54.717967 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.718015 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.718165 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0924 12:50:54.718202 5096 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0924 12:50:54.718242 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0924 12:50:54.718348 5096 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0924 12:50:54.718401 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.718456 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.718480 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.718523 5096 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0924 12:50:54.718567 5096 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0924 12:50:54.718640 5096 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0924 12:50:54.718743 5096 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:54.718793 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.718903 5096 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0924 12:50:54.718954 5096 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0924 12:50:54.718986 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.719016 5096 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:54.719072 5096 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:54.719123 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0924 12:50:54.719180 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.719203 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.719242 5096 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0924 12:50:54.719273 5096 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0924 12:50:54.719415 5096 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0924 12:50:54.719497 5096 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:54.719549 5096 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0924 12:50:54.719606 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0924 12:50:54.719629 5096 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0924 12:50:54.719675 5096 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.064s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.048s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.021s === RUN TestDiskRenderAzure I0924 12:50:54.976962 5206 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0924 12:50:54.977313 5206 disk.go:112] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0924 12:50:54.977510 5206 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:124: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0924 12:50:54.977953 5206 loadbalancer.go:132] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0924 12:50:54.978147 5206 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:102: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0924 12:50:54.978396 5206 publicipaddress.go:109] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0924 12:50:54.978608 5206 resourcegroup.go:108] Creating a new Resource Group with name: rg I0924 12:50:54.978621 5206 resourcegroup.go:110] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0924 12:50:54.978800 5206 resourcegroup.go:108] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0924 12:50:54.979258 5206 subnet.go:107] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0924 12:50:54.979456 5206 virtualnetwork.go:122] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0924 12:50:54.979637 5206 virtualnetwork.go:122] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0924 12:50:54.979909 5206 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0924 12:50:54.980141 5206 vmscaleset.go:272] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.025s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.024s === RUN TestProjectIAMBinding I0924 12:50:57.375478 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.376047 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:57.376071 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.376540 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:57.376567 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.376708 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0924 12:50:57.376817 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.376904 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:57.376924 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.377035 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:57.377056 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.377098 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0924 12:50:57.377185 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.377410 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:57.377430 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.377713 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0924 12:50:57.377737 6035 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0924 12:50:57.377865 6035 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.058s === RUN Test_OpenstackCloud_MakeCloud === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured === RUN Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value --- PASS: Test_OpenstackCloud_MakeCloud (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_false_when_not_configured (0.00s) --- PASS: Test_OpenstackCloud_MakeCloud/Ignore_volume_microversion_is_set_to_configured_value (0.00s) === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) === RUN Test_BuildClients === RUN Test_BuildClients/Empty_openstack_spec_means_no_load_balancer === RUN Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned === RUN Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned === RUN Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned --- PASS: Test_BuildClients (0.00s) --- PASS: Test_BuildClients/Empty_openstack_spec_means_no_load_balancer (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_but_no_router,_an_error_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_set,_and_there_is_a_router,_a_load-balancer_should_be_returned (0.00s) --- PASS: Test_BuildClients/When_octavia_is_not_set,_network_should_be_returned (0.00s) === RUN Test_WaitForStatusActiveIsSuccessful --- PASS: Test_WaitForStatusActiveIsSuccessful (1.00s) === RUN Test_WaitForStatusActiveResultsInInstanceNotFound --- PASS: Test_WaitForStatusActiveResultsInInstanceNotFound (1.00s) === RUN Test_WaitForStatusActiveResultsInUnableToCreateServer --- PASS: Test_WaitForStatusActiveResultsInUnableToCreateServer (1.00s) === RUN Test_WaitForStatusActiveResultsInTimeout --- PASS: Test_WaitForStatusActiveResultsInTimeout (1.00s) === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 4.409s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.037s === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.021s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.004s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.023s === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0924 12:50:59.740041 6325 bindmount.go:223] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0924 12:50:59.740103 6325 bindmount.go:232] running mount command [mount -o remount,exec /containerized_mounter] I0924 12:50:59.740116 6325 bindmount.go:223] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0924 12:50:59.740131 6325 bindmount.go:243] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0924 12:50:59.740141 6325 bindmount.go:223] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0924 12:50:59.740152 6325 bindmount.go:223] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0924 12:50:59.740163 6325 bindmount.go:223] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0924 12:50:59.740726 6325 service.go:110] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.060s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.005s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.005s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.005s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.004s === RUN TestGetProxyEnvVars W0924 12:50:59.703324 6331 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.019s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.008s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.004s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.004s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.019s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0924 12:51:03.269492 6419 s3context.go:235] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.058s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 137.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.26.6-r0.apk >>> kops: Build complete at Sun, 24 Sep 2023 12:51:09 +0000 elapsed time 0h 3m 41s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/5) Purging .makedepends-kops (20230924.124730) (2/5) Purging kubernetes (1.28.2-r0) (3/5) Purging go (1.21.1-r0) (4/5) Purging bash (5.2.15-r5) Executing bash-5.2.15-r5.pre-deinstall (5/5) Purging readline (8.2.1-r2) Executing busybox-1.36.1-r8.trigger OK: 379 MiB in 110 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...