>>> kops: Building testing/kops 1.23.0-r0 (using abuild 3.9.0-r0) started Wed, 06 Apr 2022 19:04:34 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.23.5-r1) (2/3) Installing go (1.18-r1) (3/3) Installing .makedepends-kops (20220406.190436) Executing busybox-1.35.0-r7.trigger OK: 711 MiB in 102 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.23.0.tar.gz::https://github.com/kubernetes/kops/archive/v1.23.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 134 100 134 0 0 539 0 --:--:-- --:--:-- --:--:-- 538 100 8304k 0 8304k 0 0 6840k 0 --:--:-- 0:00:01 --:--:-- 6840k 100 13.1M 0 13.1M 0 0 6090k 0 --:--:-- 0:00:02 --:--:-- 5184k 100 17.8M 0 17.8M 0 0 5692k 0 --:--:-- 0:00:03 --:--:-- 4997k 100 22.2M 0 22.2M 0 0 5395k 0 --:--:-- 0:00:04 --:--:-- 4812k 100 26.6M 0 26.6M 0 0 5224k 0 --:--:-- 0:00:05 --:--:-- 4734k 100 28.7M 0 28.7M 0 0 5142k 0 --:--:-- 0:00:05 --:--:-- 4685k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz >>> kops: Checking sha512sums... kops-1.23.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.23.0.tar.gz... go: downloading k8s.io/apimachinery v0.23.1 go: downloading github.com/aws/aws-sdk-go v1.43.11 go: downloading k8s.io/klog/v2 v2.30.0 go: downloading k8s.io/client-go v0.23.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/prometheus/client_golang v1.11.0 go: downloading k8s.io/component-base v0.23.1 go: downloading github.com/jetstack/cert-manager v1.6.1 go: downloading github.com/spf13/cobra v1.2.1 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/spf13/viper v1.8.1 go: downloading k8s.io/cli-runtime v0.23.1 go: downloading k8s.io/api v0.23.1 go: downloading google.golang.org/api v0.57.0 go: downloading github.com/blang/semver v3.5.1+incompatible go: downloading k8s.io/klog v1.0.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v0.24.0 go: downloading sigs.k8s.io/controller-runtime v0.11.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e go: downloading github.com/digitalocean/godo v1.65.0 go: downloading golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 go: downloading github.com/google/go-cmp v0.5.6 go: downloading cloud.google.com/go v0.97.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.23.1 go: downloading github.com/go-logr/logr v1.2.0 go: downloading github.com/google/go-containerregistry v0.7.0 go: downloading k8s.io/kubectl v0.23.1 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3 go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd go: downloading github.com/stretchr/testify v1.7.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/pelletier/go-toml v1.9.3 go: downloading k8s.io/kubelet v0.23.1 go: downloading k8s.io/mount-utils v0.23.1 go: downloading k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.0.2 go: downloading helm.sh/helm/v3 v3.7.1 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading gopkg.in/square/go-jose.v2 v2.5.1 go: downloading github.com/denverdino/aliyungo v0.0.0-20210425065611-55bee4942cba go: downloading github.com/Azure/azure-sdk-for-go v56.2.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.19 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.7 go: downloading github.com/fsnotify/fsnotify v1.5.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/magiconair/properties v1.8.5 go: downloading github.com/mitchellh/mapstructure v1.4.1 go: downloading github.com/spf13/afero v1.6.0 go: downloading github.com/spf13/cast v1.3.1 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/subosito/gotenv v1.2.0 go: downloading gopkg.in/ini.v1 v1.62.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.85.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.28.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.13.0 go: downloading github.com/go-ini/ini v1.62.0 go: downloading github.com/hashicorp/vault/api v1.1.1 go: downloading github.com/aliyun/alibaba-cloud-sdk-go v1.61.1059 go: downloading github.com/pkg/sftp v1.13.0 go: downloading github.com/google/go-tpm-tools v0.3.0-beta1 go: downloading github.com/google/go-tpm v0.3.2 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/hashicorp/hcl/v2 v2.10.1 go: downloading github.com/zclconf/go-cty v1.8.2 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/go-querystring v1.0.0 go: downloading github.com/googleapis/gnostic v0.5.5 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.27.1 go: downloading golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.23.1 go: downloading github.com/docker/cli v20.10.10+incompatible go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/russross/blackfriday v1.5.2 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.1.4 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.1 go: downloading github.com/prometheus/procfs v0.6.0 go: downloading sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 go: downloading k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.1 go: downloading k8s.io/csi-translation-lib v0.23.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.14 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.2 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading golang.org/x/text v0.3.7 go: downloading google.golang.org/grpc v1.42.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.1.1 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading k8s.io/apiextensions-apiserver v0.23.1 go: downloading sigs.k8s.io/kustomize/api v0.10.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.0 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.0.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.1 go: downloading github.com/hashicorp/go-multierror v1.1.0 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/vault/sdk v0.2.1 go: downloading github.com/kr/fs v0.1.0 go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 go: downloading github.com/json-iterator/go v1.1.12 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/docker/distribution v2.7.1+incompatible go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.17.0 go: downloading go.uber.org/goleak v1.1.12 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/smartystreets/goconvey v1.6.4 go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.1 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/hashicorp/go-hclog v0.16.1 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.10.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/moby/term v0.0.0-20210610120745-9d4ed1856297 go: downloading github.com/ghodss/yaml v1.0.0 go: downloading github.com/stretchr/objx v0.2.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/armon/go-metrics v0.3.3 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.35 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/form3tech-oss/jwt-go v3.2.3+incompatible go: downloading github.com/googleapis/gax-go/v2 v2.1.0 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/mitchellh/reflectwalk v1.0.1 go: downloading github.com/google/go-attestation v0.3.2 go: downloading github.com/docker/docker v20.10.10+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/opencontainers/image-spec v1.0.2-0.20210730191737-8e42a01fb1b7 go: downloading github.com/fatih/color v1.12.0 go: downloading github.com/mattn/go-colorable v0.1.8 go: downloading github.com/mattn/go-isatty v0.0.12 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/containerd/containerd v1.5.7 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading gotest.tools v2.2.0+incompatible go: downloading oras.land/oras-go v0.4.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20210804104954-38ab4c606ee3 go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 go: downloading github.com/fvbommel/sortorder v1.0.1 go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/klauspost/compress v1.13.6 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/hashicorp/go-immutable-radix v1.1.0 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading google.golang.org/genproto v0.0.0-20211111162719-482062a4217b go: downloading go.uber.org/atomic v1.7.0 go: downloading go.uber.org/multierr v1.6.0 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.1 go: downloading github.com/golang/snappy v0.0.3 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/google/certificate-transparency-go v1.1.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/jtolds/gls v4.20.0+incompatible go: downloading github.com/smartystreets/assertions v1.1.0 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading rsc.io/letsencrypt v0.0.3 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.8.22 go: downloading github.com/moby/sys/mountinfo v0.4.1 go: downloading github.com/Microsoft/go-winio v0.5.1 go: downloading golang.org/x/mod v0.5.1 go: downloading github.com/containerd/cgroups v1.0.1 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/frankban/quicktest v1.11.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 go: downloading github.com/Masterminds/sprig v2.22.0+incompatible go: downloading github.com/cpuguy83/go-md2man v1.0.10 go: downloading github.com/Masterminds/semver v1.5.0 go: downloading github.com/cespare/xxhash v1.1.0 go: downloading github.com/apparentlymart/go-textseg v1.0.0 /bin/sh: tools/get_version.sh: not found go build -ldflags=all="-X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220328-482-g5db24de469 " -o /home/buildozer/aports/testing/kops/src/kops-1.23.0/.build/local/kops k8s.io/kops/cmd/kops/ go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0406 19:06:37.963704 8829 addon.go:218] addon test wants to update all nodes I0406 19:06:37.965794 8829 addon.go:218] addon test wants to update worker nodes I0406 19:06:37.966151 8829 addon.go:218] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0406 19:06:37.966579 8829 addon.go:251] installing PKI for "test" I0406 19:06:38.712670 8829 addon.go:251] installing PKI for "test" --- PASS: Test_InstallPKI (1.27s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.415s ? k8s.io/kops/channels/pkg/cmd [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0406 19:06:55.301271 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.301371 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.301424 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.301485 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.301532 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.301558 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.301581 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.301631 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.301647 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.301700 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.301731 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.493746 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.493919 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.494594 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.518807 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.518867 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.518915 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.518968 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.519014 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.519044 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.519074 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.519108 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.519138 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.519207 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.519247 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.559088 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.559172 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.559440 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.565154 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.565185 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.565212 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.565239 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.565262 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.565279 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.565296 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.565317 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.565333 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.565372 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.565395 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.612958 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.613047 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.613325 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.618991 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.619023 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.619051 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.619080 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.619105 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.619121 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.619139 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.619160 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.619175 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.619222 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.619252 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.687510 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.688049 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.688345 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.694398 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.694430 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.694463 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.694492 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.694518 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.694534 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.694554 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.694574 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.694588 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.694627 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.694650 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.727566 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.727646 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.727900 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.734130 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.734171 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.734200 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.734229 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.734255 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.734273 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.734290 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.734312 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.734327 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.734363 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.734386 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.802952 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.803038 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.803295 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.51s) === RUN TestCreateClusterOverride I0406 19:06:55.810826 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.810904 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.810969 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.811024 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.811072 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.811103 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.811135 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.811175 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.811206 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.811306 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.811352 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.859170 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.859999 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.860669 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.06s) === RUN TestCreateClusterComplex I0406 19:06:55.875095 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.875151 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.875211 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.875272 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.875322 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.875365 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.875418 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.875472 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.875507 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.875617 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.875672 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.906644 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.906736 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.907055 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.04s) === RUN TestCreateClusterHA I0406 19:06:55.912873 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.912909 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.912936 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.912967 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.912997 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.913014 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.913032 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.913060 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.913079 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.913147 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.913172 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.937007 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:55.937060 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:55.937073 12299 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 19:06:55.937088 12299 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0406 19:06:55.937291 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.948730 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.948791 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.948833 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.948872 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.948908 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.948930 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.948954 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.948979 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.949001 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.949054 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:55.949085 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.048297 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.048386 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.048400 12299 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 19:06:56.048410 12299 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0406 19:06:56.048703 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.14s) === RUN TestCreateClusterHAGCE I0406 19:06:56.057792 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.057821 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.057846 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.057874 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.057899 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.057917 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.057936 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.057957 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.057973 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.058032 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.058058 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0406 19:06:56.115681 12299 new_cluster.go:372] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0406 19:06:56.115707 12299 new_cluster.go:373] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0406 19:06:56.115775 12299 new_cluster.go:1102] Cloud Provider ID = gce I0406 19:06:56.115864 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.07s) === RUN TestCreateClusterGCE I0406 19:06:56.125481 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.125513 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.125542 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.125570 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.125594 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.125609 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.125625 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.125643 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.125660 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.125705 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.125728 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.196348 12299 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0406 19:06:56.196416 12299 new_cluster.go:1102] Cloud Provider ID = gce I0406 19:06:56.196459 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.08s) === RUN TestCreateClusterHASharedZone I0406 19:06:56.202409 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.202442 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.202470 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.202500 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.202526 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.202543 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.202562 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.202582 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.202600 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.202647 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.202671 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0406 19:06:56.244960 12299 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0406 19:06:56.245020 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.245100 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.245455 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.05s) === RUN TestCreateClusterHASharedZones I0406 19:06:56.254507 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.254547 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.254582 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.254618 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.254651 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.254670 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.254691 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.254719 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.254752 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.254819 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.254853 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0406 19:06:56.308112 12299 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0406 19:06:56.308150 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.308198 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.308211 12299 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 19:06:56.308428 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.06s) === RUN TestCreateClusterPrivate I0406 19:06:56.318362 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.318392 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.318414 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.318438 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.318457 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.318474 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.318493 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.318509 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.318523 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.318564 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.318586 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.377703 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.377821 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.377845 12299 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 19:06:56.378235 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:56.378290 12299 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.07s) === RUN TestCreateClusterPrivateGCE I0406 19:06:56.387528 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.387561 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.387595 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.387630 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.387659 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.387679 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.387700 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.387722 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.387741 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.387793 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.387822 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.433483 12299 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0406 19:06:56.433527 12299 new_cluster.go:1102] Cloud Provider ID = gce I0406 19:06:56.433567 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.05s) === RUN TestCreateClusterWithNGWSpecified I0406 19:06:56.439935 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.439960 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.439982 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.440008 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.440028 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.440044 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.440060 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.440076 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.440090 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.440127 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.440148 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.461845 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.461894 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.461907 12299 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 19:06:56.462092 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:56.462117 12299 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.03s) === RUN TestCreateClusterWithINGWSpecified I0406 19:06:56.472979 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.473024 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.473067 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.473113 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.473148 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.473177 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.473206 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.473232 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.473257 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.473328 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.473368 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.541337 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.541396 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.541427 12299 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 19:06:56.541658 12299 zones.go:93] ListHostedZonesPages { } I0406 19:06:56.541688 12299 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.08s) === RUN TestCreateClusterSharedVPC I0406 19:06:56.548342 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.548367 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.548391 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.548422 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.548443 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.548458 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.548476 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.548492 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.548508 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.548548 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.548571 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.617272 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.617347 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.617399 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.617442 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.617467 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.617520 12299 subnets.go:182] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0406 19:06:56.617830 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.08s) === RUN TestCreateClusterSharedSubnets I0406 19:06:56.626896 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.626963 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.627018 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.627071 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.627114 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.627171 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.627207 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.627244 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.627273 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.627359 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.627402 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.664958 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.665011 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.665096 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.665148 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.665169 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.665197 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.665213 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.665541 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.05s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0406 19:06:56.671919 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.671949 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.671976 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.672006 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.672032 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.672052 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.672071 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.672092 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.672108 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.672161 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.672187 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.717759 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:56.717805 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.717826 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.717898 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.717949 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.717968 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.717997 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.718011 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.718323 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.05s) === RUN TestCreateClusterPrivateSharedSubnets I0406 19:06:56.724095 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.724124 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.724147 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.724175 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.724205 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.724225 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.724244 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.724271 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.724287 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.724340 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.724365 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.767152 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.767187 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.767478 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.767696 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.767737 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.767806 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.767828 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.767862 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:06:56.767882 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 19:06:56.768297 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.05s) === RUN TestCreateClusterIPv6 I0406 19:06:56.775353 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.775386 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.775414 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.775445 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.775475 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.775492 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.775512 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.775533 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.775552 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.775608 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.775633 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 19:06:56.809781 12299 new_cluster.go:1102] Cloud Provider ID = aws I0406 19:06:56.809870 12299 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 19:06:56.810244 12299 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.04s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0406 19:06:56.816664 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:56.816700 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:56.816729 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:56.816760 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:56.816842 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:56.816862 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:56.816881 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:56.816904 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:56.816922 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:56.833038 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:06:56.834670 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:06:56.834758 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 19:06:58.968292 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:01.223143 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:01.223870 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:01.224919 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:01.225571 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:01.226357 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:01.226414 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:01.227576 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:01.229641 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:01.236699 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:01.237955 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.238510 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.239589 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.240467 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.240620 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:01.241201 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:01.274511 12299 target.go:112] Terraform output is in /tmp/test1168510510/out I0406 19:07:01.278662 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:01.278718 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:01.278766 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:01.278818 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:01.278862 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:01.278890 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:01.278919 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:01.278949 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:01.278977 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:01.292660 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:01.293890 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.293941 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 19:07:01.336726 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:01.424299 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:01.424639 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:01.424656 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:01.424721 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:01.424909 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:01.424988 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0406 19:07:01.425009 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:01.424997 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:01.425009 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:01.425051 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:01.429305 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:01.431811 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:01.432163 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.432325 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.433253 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.433425 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.433550 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:01.433773 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:01.438595 12299 target.go:168] Cloudformation output is in /tmp/test621211287/out --- PASS: TestMinimal (4.63s) === RUN TestMinimal_v1_23 I0406 19:07:01.450157 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:01.450207 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:01.450247 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:01.450291 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:01.450369 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:01.450400 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:01.450430 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:01.450461 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:01.450490 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:01.468213 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* I0406 19:07:01.469555 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:01.469608 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:02.768746 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:02.866856 12299 executor.go:111] Tasks: 0 done / 91 total; 46 can run W0406 19:07:02.867622 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:02.867705 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:02.867738 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:02.867814 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:02.867826 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:02.867939 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:02.870409 12299 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0406 19:07:02.877799 12299 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0406 19:07:02.877991 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:02.878303 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:02.878617 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:02.879767 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:02.879817 12299 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0406 19:07:02.879935 12299 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0406 19:07:02.910207 12299 target.go:112] Terraform output is in /tmp/test1325172323/out I0406 19:07:02.914611 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:02.914675 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:02.914725 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:02.914800 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:02.914845 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:02.914874 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:02.914905 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:02.914942 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:02.914973 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:02.937914 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:02.940675 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:02.940768 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 19:07:02.979119 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:03.055711 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:03.056039 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:03.056166 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0406 19:07:03.056186 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:03.056195 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:03.056182 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:03.056267 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:03.056084 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:03.056391 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:03.056467 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:03.058607 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:03.063166 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:03.063411 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.063626 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.063994 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.065388 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.065469 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:03.065629 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:03.067304 12299 target.go:168] Cloudformation output is in /tmp/test1467563595/out --- PASS: TestMinimal_v1_23 (1.62s) === RUN TestNvidia I0406 19:07:03.073168 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:03.073198 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:03.073223 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:03.073251 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:03.073299 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:03.073315 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:03.073333 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:03.073352 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:03.073369 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:03.089664 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:03.091645 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.091701 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.129175 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:03.226727 12299 executor.go:111] Tasks: 0 done / 87 total; 45 can run W0406 19:07:03.227160 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:03.227279 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:03.227371 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:03.227389 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:03.227444 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:03.227500 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:03.229480 12299 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0406 19:07:03.232566 12299 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0406 19:07:03.232737 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.233029 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.237142 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.237271 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.237361 12299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 19:07:03.237637 12299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 19:07:03.284816 12299 target.go:112] Terraform output is in /tmp/test1968065242/out I0406 19:07:03.287406 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:03.287442 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:03.287471 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:03.287505 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:03.287530 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:03.287546 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:03.287564 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:03.287584 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:03.287600 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:03.301074 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:03.302655 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.302712 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.340595 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:03.431823 12299 executor.go:111] Tasks: 0 done / 87 total; 45 can run W0406 19:07:03.432354 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:03.432436 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:03.432503 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:03.432467 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:03.432563 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:03.432596 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:03.432655 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:03.432667 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:03.433287 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:03.437834 12299 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0406 19:07:03.442645 12299 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0406 19:07:03.442877 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.443156 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.444097 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.445489 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.445592 12299 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 19:07:03.445818 12299 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 19:07:03.447982 12299 target.go:168] Cloudformation output is in /tmp/test1654976448/out --- PASS: TestNvidia (0.38s) === RUN TestMinimalGossip I0406 19:07:03.457778 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:03.457832 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:03.457882 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:03.457933 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:03.457992 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:03.458021 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:03.458052 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:03.458109 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:03.458147 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:03.483825 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.483931 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.522412 12299 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0406 19:07:03.628223 12299 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0406 19:07:03.628825 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:03.629103 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:03.629134 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:03.632862 12299 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0406 19:07:03.640496 12299 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0406 19:07:03.641021 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.641198 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.642362 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.643812 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.643933 12299 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0406 19:07:03.644250 12299 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0406 19:07:03.698273 12299 target.go:112] Terraform output is in /tmp/test340950521/out --- PASS: TestMinimalGossip (0.24s) === RUN TestMinimalGossipIRSA I0406 19:07:03.702521 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:03.702581 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:03.702635 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:03.702687 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:03.702730 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:03.702759 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:03.702833 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:03.702868 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:03.702895 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:03.730620 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.730714 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.770892 12299 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0406 19:07:03.890920 12299 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0406 19:07:03.891557 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:03.891841 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:03.892032 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:03.894547 12299 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0406 19:07:03.903172 12299 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0406 19:07:03.903504 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.903643 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.904773 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.905748 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:03.905866 12299 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0406 19:07:03.906155 12299 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0406 19:07:03.960704 12299 target.go:112] Terraform output is in /tmp/test1161776563/out --- PASS: TestMinimalGossipIRSA (0.26s) === RUN TestMinimalGCE I0406 19:07:03.965316 12299 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0406 19:07:03.967139 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0406 19:07:04.859986 12299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 19:07:04.860081 12299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 19:07:04.860226 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0406 19:07:04.860272 12299 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0406 19:07:04.860295 12299 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0406 19:07:04.860314 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:04.860388 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:04.860413 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:04.861998 12299 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0406 19:07:04.865418 12299 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0406 19:07:04.874355 12299 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0406 19:07:04.875965 12299 instancetemplate.go:555] adding csa: default I0406 19:07:04.877088 12299 instancetemplate.go:555] adding csa: default I0406 19:07:04.877205 12299 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0406 19:07:04.877497 12299 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0406 19:07:04.909705 12299 target.go:112] Terraform output is in /tmp/test2635126763/out --- PASS: TestMinimalGCE (0.95s) === RUN TestMinimalGCEPrivate I0406 19:07:04.913831 12299 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0406 19:07:04.915686 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0406 19:07:05.746454 12299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 19:07:05.746542 12299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 19:07:05.746679 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0406 19:07:05.746721 12299 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0406 19:07:05.746744 12299 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0406 19:07:05.746763 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:05.746839 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:05.746863 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:05.748539 12299 executor.go:111] Tasks: 0 done / 63 total; 37 can run I0406 19:07:05.751996 12299 executor.go:111] Tasks: 37 done / 63 total; 18 can run I0406 19:07:05.759344 12299 executor.go:111] Tasks: 55 done / 63 total; 4 can run I0406 19:07:05.761783 12299 instancetemplate.go:555] adding csa: default I0406 19:07:05.761901 12299 executor.go:111] Tasks: 59 done / 63 total; 3 can run I0406 19:07:05.763180 12299 instancetemplate.go:555] adding csa: default I0406 19:07:05.763300 12299 executor.go:111] Tasks: 62 done / 63 total; 1 can run I0406 19:07:05.763446 12299 executor.go:111] Tasks: 63 done / 63 total; 0 can run I0406 19:07:05.794792 12299 target.go:112] Terraform output is in /tmp/test2137850515/out --- PASS: TestMinimalGCEPrivate (0.88s) === RUN TestHA I0406 19:07:05.798951 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:05.799013 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:05.799068 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:05.799134 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:05.799182 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:05.799211 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:05.799243 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:05.799280 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:05.799309 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:05.831560 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:05.834216 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.834313 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.834367 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.834417 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.873635 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:05.985554 12299 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0406 19:07:05.986023 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:05.986180 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:05.986410 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:05.986431 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:05.986437 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:05.986454 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:05.986535 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:05.986612 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:05.989058 12299 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0406 19:07:05.998943 12299 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0406 19:07:05.999175 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.999424 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.999650 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:05.999935 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.000404 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.001702 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.002010 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.002368 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.002495 12299 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0406 19:07:06.002922 12299 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0406 19:07:06.082345 12299 target.go:112] Terraform output is in /tmp/test830905473/out --- PASS: TestHA (0.29s) === RUN TestHighAvailabilityGCE I0406 19:07:06.087177 12299 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0406 19:07:06.089224 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0406 19:07:06.289400 12299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 19:07:06.289493 12299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 19:07:06.289624 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0406 19:07:06.289665 12299 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0406 19:07:06.289688 12299 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0406 19:07:06.289708 12299 autoscalinggroup.go:168] gsa: default W0406 19:07:06.289765 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0406 19:07:06.289795 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:06.289815 12299 autoscalinggroup.go:168] gsa: default W0406 19:07:06.289865 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0406 19:07:06.289894 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:06.289916 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:06.289982 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:06.290005 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:06.292078 12299 executor.go:111] Tasks: 0 done / 76 total; 41 can run I0406 19:07:06.299151 12299 executor.go:111] Tasks: 41 done / 76 total; 20 can run I0406 19:07:06.304840 12299 executor.go:111] Tasks: 61 done / 76 total; 7 can run I0406 19:07:06.307416 12299 instancetemplate.go:555] adding csa: default I0406 19:07:06.307440 12299 instancetemplate.go:555] adding csa: default I0406 19:07:06.307554 12299 instancetemplate.go:555] adding csa: default I0406 19:07:06.308590 12299 executor.go:111] Tasks: 68 done / 76 total; 5 can run I0406 19:07:06.312625 12299 instancetemplate.go:555] adding csa: default I0406 19:07:06.312766 12299 executor.go:111] Tasks: 73 done / 76 total; 3 can run I0406 19:07:06.313075 12299 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0406 19:07:06.353136 12299 target.go:112] Terraform output is in /tmp/test1642389413/out --- PASS: TestHighAvailabilityGCE (0.27s) === RUN TestComplex I0406 19:07:06.357967 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:06.358029 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:06.358128 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:06.358187 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:06.358285 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:06.358320 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:06.358352 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:06.358390 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:06.358420 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:06.392932 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:06.396548 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.396662 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.437205 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:06.552246 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0406 19:07:06.553164 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:06.553237 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:06.553327 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:06.553269 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:06.553397 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:06.555371 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:06.564638 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:06.565308 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.565516 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.566780 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.568326 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.568454 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:06.568905 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 19:07:06.644844 12299 target.go:112] Terraform output is in /tmp/test1886882700/out I0406 19:07:06.648855 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:06.648922 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:06.648976 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:06.649032 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:06.649078 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:06.649108 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:06.649141 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:06.649177 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:06.649207 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:06.680389 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:06.683957 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.684055 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.722362 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:06.834690 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0406 19:07:06.834983 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:06.835564 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:06.835622 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:06.835607 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:06.836444 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:06.843172 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:06.848657 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:06.849262 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.849533 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.850604 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.851920 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.852146 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:06.852626 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 19:07:06.859871 12299 target.go:168] Cloudformation output is in /tmp/test3118377353/out I0406 19:07:06.873367 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:06.873423 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:06.873471 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:06.873519 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:06.873559 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:06.873588 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:06.873619 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:06.873650 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:06.873677 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:06.904486 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:06.907972 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.908061 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:06.945796 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:07.038782 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0406 19:07:07.039704 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.039787 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:07.039846 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.039825 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:07.039966 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:07.041823 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:07.047113 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:07.047696 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.049145 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.052254 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.053466 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.053607 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:07.053985 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 19:07:07.128114 12299 target.go:112] Terraform output is in /tmp/test3034122502/out --- PASS: TestComplex (0.77s) === RUN TestCompress I0406 19:07:07.132291 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:07.132349 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:07.132402 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:07.132455 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:07.132498 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:07.132527 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:07.132556 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:07.132588 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:07.132614 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:07.160261 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:07.163081 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.163185 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.202859 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:07.314200 12299 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0406 19:07:07.314859 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.314932 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:07.314970 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:07.315101 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:07.315137 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.317246 12299 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0406 19:07:07.326084 12299 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0406 19:07:07.326488 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.326629 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.329461 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.331250 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.331533 12299 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0406 19:07:07.332391 12299 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0406 19:07:07.380505 12299 target.go:112] Terraform output is in /tmp/test876615566/out --- PASS: TestCompress (0.25s) === RUN TestExternalPolicies I0406 19:07:07.385435 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:07.385510 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:07.385587 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:07.385665 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:07.385726 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:07.385774 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:07.385821 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:07.385872 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:07.385914 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:07.417120 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:07.420404 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.420504 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.458038 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:07.569794 12299 executor.go:111] Tasks: 0 done / 99 total; 46 can run I0406 19:07:07.570075 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:07.570259 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:07.570293 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:07.570385 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:07.570624 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:07.570941 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.577167 12299 executor.go:111] Tasks: 46 done / 99 total; 20 can run I0406 19:07:07.582236 12299 executor.go:111] Tasks: 66 done / 99 total; 30 can run I0406 19:07:07.582686 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.582952 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.583800 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.585336 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.585459 12299 executor.go:111] Tasks: 96 done / 99 total; 3 can run I0406 19:07:07.585819 12299 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0406 19:07:07.651765 12299 target.go:112] Terraform output is in /tmp/test2244717132/out --- PASS: TestExternalPolicies (0.27s) === RUN TestMinimalIPv6 I0406 19:07:07.655978 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:07.656042 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:07.656096 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:07.656152 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:07.656251 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:07.656282 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:07.656313 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:07.656352 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:07.656382 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:07.681456 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:07.684562 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.684662 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.722887 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:07.901768 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0406 19:07:07.902199 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:07.902281 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.902238 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:07.902392 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:07.902516 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:07.902675 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:07.904529 12299 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0406 19:07:07.910106 12299 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0406 19:07:07.910396 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.910626 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.911893 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.913106 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:07.913262 12299 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0406 19:07:07.913853 12299 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0406 19:07:07.914216 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:07.981322 12299 target.go:112] Terraform output is in /tmp/test3184168682/out I0406 19:07:07.985762 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:07.985820 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:07.985871 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:07.985921 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:07.985959 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:07.985987 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:07.986017 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:07.986060 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:07.986089 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:08.010812 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:08.013998 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.014086 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.050912 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:08.228634 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0406 19:07:08.229278 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:08.229395 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:08.229453 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:08.229483 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:08.229652 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:08.229629 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:08.231840 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 19:07:08.229708 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:08.232170 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:08.235896 12299 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0406 19:07:08.241436 12299 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0406 19:07:08.242051 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.242322 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.243378 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.244916 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.245083 12299 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0406 19:07:08.245648 12299 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0406 19:07:08.245965 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:08.249878 12299 target.go:168] Cloudformation output is in /tmp/test3579479617/out --- PASS: TestMinimalIPv6 (0.61s) === RUN TestMinimalIPv6Private I0406 19:07:08.264062 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:08.264120 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:08.264171 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:08.264226 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:08.264322 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:08.264351 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:08.264381 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:08.264417 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:08.264447 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:08.292762 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:08.296218 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.296316 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.334508 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:08.512510 12299 executor.go:111] Tasks: 0 done / 118 total; 46 can run I0406 19:07:08.513187 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:08.513264 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:08.513226 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:08.513338 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:08.513469 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:08.513575 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:08.515810 12299 executor.go:111] Tasks: 46 done / 118 total; 23 can run I0406 19:07:08.520898 12299 executor.go:111] Tasks: 69 done / 118 total; 34 can run I0406 19:07:08.521559 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.522863 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.527276 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.528804 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.528945 12299 executor.go:111] Tasks: 103 done / 118 total; 9 can run I0406 19:07:08.529556 12299 executor.go:111] Tasks: 112 done / 118 total; 6 can run I0406 19:07:08.530006 12299 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0406 19:07:08.599193 12299 target.go:112] Terraform output is in /tmp/test1252776060/out --- PASS: TestMinimalIPv6Private (0.34s) === RUN TestMinimalIPv6Calico I0406 19:07:08.603568 12299 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0406 19:07:08.603805 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:08.603864 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:08.603913 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:08.603968 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:08.604065 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:08.604099 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:08.604131 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:08.604168 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:08.604195 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:08.633543 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:08.636826 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.636927 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:08.676551 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:09.092332 12299 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0406 19:07:09.093639 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:09.093887 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:09.094100 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:09.094974 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:09.095023 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:09.095155 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:09.099161 12299 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0406 19:07:09.106248 12299 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0406 19:07:09.106554 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.106980 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.110488 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.112494 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.112649 12299 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0406 19:07:09.113262 12299 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0406 19:07:09.113756 12299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0406 19:07:09.175664 12299 target.go:112] Terraform output is in /tmp/test3750581457/out I0406 19:07:09.181049 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:09.181114 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:09.181168 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:09.181222 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:09.181263 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:09.181292 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:09.181320 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:09.181355 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:09.181383 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:09.210473 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:09.213721 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.213823 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.252103 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:09.660555 12299 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0406 19:07:09.660970 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:09.661098 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:09.661320 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:09.661349 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:09.661496 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:09.661705 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:09.661900 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:09.661929 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:09.662064 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:09.663438 12299 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0406 19:07:09.672392 12299 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0406 19:07:09.673057 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.673299 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.674633 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.675753 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.675910 12299 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0406 19:07:09.676552 12299 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0406 19:07:09.677004 12299 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0406 19:07:09.680876 12299 target.go:168] Cloudformation output is in /tmp/test3105760347/out I0406 19:07:09.692281 12299 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Calico (1.09s) === RUN TestMinimalIPv6Cilium I0406 19:07:09.692385 12299 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0406 19:07:09.692587 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:09.692634 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:09.692675 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:09.692721 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:09.692800 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:09.692827 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:09.692855 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:09.692887 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:09.692913 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:09.722064 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:09.725494 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.725592 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:09.762712 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:09.990364 12299 executor.go:111] Tasks: 0 done / 106 total; 47 can run I0406 19:07:09.990950 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:09.990984 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:09.991025 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:09.991001 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:09.991103 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:09.994244 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:09.998227 12299 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0406 19:07:10.004184 12299 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0406 19:07:10.004859 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.005159 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.006242 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.007899 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.008049 12299 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0406 19:07:10.008597 12299 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0406 19:07:10.008921 12299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0406 19:07:10.076735 12299 target.go:112] Terraform output is in /tmp/test2597161233/out I0406 19:07:10.081251 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:10.081317 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:10.081369 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:10.081445 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:10.081495 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:10.081523 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:10.081554 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:10.081597 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:10.081628 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:10.107114 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:10.110649 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.110752 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.148907 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:10.371567 12299 executor.go:111] Tasks: 0 done / 106 total; 47 can run W0406 19:07:10.371953 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:10.372160 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:10.372526 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:10.372735 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:10.372751 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:10.372779 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:10.372872 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:10.372978 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:10.372976 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0406 19:07:10.374316 12299 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0406 19:07:10.379870 12299 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0406 19:07:10.380336 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.380546 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.381639 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.383103 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.383261 12299 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0406 19:07:10.383696 12299 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0406 19:07:10.384022 12299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0406 19:07:10.388334 12299 target.go:168] Cloudformation output is in /tmp/test1524469300/out I0406 19:07:10.403145 12299 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Cilium (0.71s) === RUN TestMinimalWarmPool I0406 19:07:10.403536 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:10.403603 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:10.403653 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:10.403703 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:10.403784 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:10.403815 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:10.403845 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:10.403876 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:10.403903 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:10.428349 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:10.431590 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.431677 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.471997 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:10.681759 12299 executor.go:111] Tasks: 0 done / 89 total; 46 can run W0406 19:07:10.682374 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:10.682526 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:10.682769 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:10.682800 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:10.682794 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:10.682969 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:10.684673 12299 executor.go:111] Tasks: 46 done / 89 total; 19 can run I0406 19:07:10.689812 12299 executor.go:111] Tasks: 65 done / 89 total; 21 can run I0406 19:07:10.690314 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.690521 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.691843 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.692890 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.693006 12299 executor.go:111] Tasks: 86 done / 89 total; 2 can run I0406 19:07:10.693321 12299 executor.go:111] Tasks: 88 done / 89 total; 1 can run I0406 19:07:10.693698 12299 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0406 19:07:10.749192 12299 target.go:112] Terraform output is in /tmp/test3328033042/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I0406 19:07:10.753876 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:10.753931 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:10.753979 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:10.754028 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:10.754109 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:10.754139 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:10.754169 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:10.754204 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:10.754231 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:10.783097 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:10.786043 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.786298 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.825336 12299 zones.go:93] ListHostedZonesPages { } W0406 19:07:10.924969 12299 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0406 19:07:10.925027 12299 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0406 19:07:10.926908 12299 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0406 19:07:10.927064 12299 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0406 19:07:10.927102 12299 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:10.939699 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0406 19:07:10.940106 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:10.940376 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:10.940414 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:10.940497 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:10.940641 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:10.940817 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:10.940983 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:10.941005 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:10.941127 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0406 19:07:10.943138 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:10.951798 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:10.952117 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.952338 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.953209 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.955424 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.955567 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:10.955836 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:10.959007 12299 target.go:168] Cloudformation output is in /tmp/test1341674914/out --- PASS: TestMinimalEtcd (0.22s) === RUN TestMinimalGp3 I0406 19:07:10.970230 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:10.970277 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:10.970321 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:10.970368 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:10.970446 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:10.970477 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:10.970508 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:10.970540 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:10.970566 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:10.994983 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:10.997735 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:10.997824 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.034991 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:11.148706 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0406 19:07:11.149190 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:11.150563 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:11.151851 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:11.152420 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:11.152470 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:11.152633 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:11.156401 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:11.161573 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:11.162057 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.162262 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.163423 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.164670 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.164859 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:11.165412 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:11.221620 12299 target.go:112] Terraform output is in /tmp/test2350285888/out I0406 19:07:11.225776 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:11.225834 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:11.225884 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:11.225933 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:11.225974 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:11.226002 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:11.226031 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:11.226061 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:11.226089 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:11.250813 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:11.253657 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.253752 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.293206 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:11.403322 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0406 19:07:11.404158 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:11.404168 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:11.404402 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:11.404419 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:11.404435 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:11.404553 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:11.404560 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:11.404576 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:11.404664 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:11.406916 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:11.415473 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:11.415787 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.415926 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.416952 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.418400 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.418536 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:11.418818 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:11.421761 12299 target.go:168] Cloudformation output is in /tmp/test909038451/out --- PASS: TestMinimalGp3 (0.46s) === RUN TestExistingIAMCloudformation I0406 19:07:11.433117 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:11.433165 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:11.433211 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:11.433258 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:11.433333 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:11.433362 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:11.433391 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:11.433424 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:11.433450 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:11.456707 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:11.459371 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.459455 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.496579 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:11.607384 12299 executor.go:111] Tasks: 0 done / 76 total; 42 can run I0406 19:07:11.607723 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:11.607834 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:11.607854 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:11.608061 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:11.608128 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:11.608248 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:11.607975 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0406 19:07:11.608280 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:11.608441 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:11.615130 12299 executor.go:111] Tasks: 42 done / 76 total; 11 can run I0406 19:07:11.620238 12299 executor.go:111] Tasks: 53 done / 76 total; 21 can run I0406 19:07:11.620440 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.620643 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.621712 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.622756 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.622891 12299 executor.go:111] Tasks: 74 done / 76 total; 2 can run I0406 19:07:11.623161 12299 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0406 19:07:11.625381 12299 target.go:168] Cloudformation output is in /tmp/test3411419258/out --- PASS: TestExistingIAMCloudformation (0.20s) === RUN TestExistingSG I0406 19:07:11.634498 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:11.634547 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:11.634591 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:11.634638 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:11.634681 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:11.634710 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:11.634760 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:11.634795 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:11.634822 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:11.671380 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:11.674508 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.674600 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.674660 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.674718 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.714039 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:11.830445 12299 executor.go:111] Tasks: 0 done / 139 total; 50 can run I0406 19:07:11.830927 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:11.831756 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:11.830966 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:11.833048 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:11.834462 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:11.834515 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:11.835096 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:11.835336 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:11.837928 12299 executor.go:111] Tasks: 50 done / 139 total; 26 can run I0406 19:07:11.847100 12299 executor.go:111] Tasks: 76 done / 139 total; 58 can run I0406 19:07:11.847530 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.847859 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.848111 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.848287 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.851071 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.854113 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.854307 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.854708 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.854877 12299 executor.go:111] Tasks: 134 done / 139 total; 5 can run I0406 19:07:11.855333 12299 executor.go:111] Tasks: 139 done / 139 total; 0 can run I0406 19:07:11.948527 12299 target.go:112] Terraform output is in /tmp/test4172592618/out --- PASS: TestExistingSG (0.32s) === RUN TestBastionAdditionalUserData I0406 19:07:11.955863 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:11.955931 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:11.955985 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:11.956037 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:11.956092 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:11.956121 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:11.956151 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:11.956184 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:11.956213 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:11.987457 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:11.990500 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.990604 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:11.990669 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.028818 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:12.146037 12299 executor.go:111] Tasks: 0 done / 120 total; 49 can run W0406 19:07:12.146897 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:12.146914 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:12.146966 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:12.147127 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:12.147231 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:12.147265 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:12.147386 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:12.149339 12299 executor.go:111] Tasks: 49 done / 120 total; 29 can run I0406 19:07:12.154513 12299 executor.go:111] Tasks: 78 done / 120 total; 36 can run I0406 19:07:12.155799 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.156204 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.156439 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.156870 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.160270 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.161205 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.161336 12299 executor.go:111] Tasks: 114 done / 120 total; 5 can run I0406 19:07:12.161844 12299 executor.go:111] Tasks: 119 done / 120 total; 1 can run I0406 19:07:12.162016 12299 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0406 19:07:12.237470 12299 target.go:112] Terraform output is in /tmp/test2224241716/out --- PASS: TestBastionAdditionalUserData (0.29s) === RUN TestPrivateWeave I0406 19:07:12.244127 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:12.244193 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:12.244251 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:12.244308 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:12.244400 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:12.244430 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:12.244460 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:12.244498 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:12.244528 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:12.272670 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:12.275642 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.275738 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.275807 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.313876 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:12.447007 12299 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0406 19:07:12.447519 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:12.448564 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:12.448810 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:12.448812 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:12.448976 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:12.448878 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:12.449039 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:12.453864 12299 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0406 19:07:12.458946 12299 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0406 19:07:12.459444 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.459618 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.459849 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.459969 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.462537 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.466640 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.466776 12299 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0406 19:07:12.467203 12299 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0406 19:07:12.467367 12299 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0406 19:07:12.541129 12299 target.go:112] Terraform output is in /tmp/test2921818704/out --- PASS: TestPrivateWeave (0.30s) === RUN TestPrivateFlannel I0406 19:07:12.546045 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:12.546104 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:12.546161 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:12.546211 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:12.546299 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:12.546329 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:12.546359 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:12.546393 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:12.546420 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:12.574270 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:12.577107 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.577190 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.577249 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.615037 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:12.742326 12299 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0406 19:07:12.742693 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:12.742736 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:12.742851 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:12.743036 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:12.743089 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:12.743184 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:12.743364 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:12.745774 12299 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0406 19:07:12.751513 12299 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0406 19:07:12.751957 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.752262 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.753285 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.753536 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.757796 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.759194 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.759330 12299 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0406 19:07:12.759813 12299 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0406 19:07:12.759989 12299 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0406 19:07:12.831495 12299 target.go:112] Terraform output is in /tmp/test2814803104/out --- PASS: TestPrivateFlannel (0.29s) === RUN TestPrivateCalico I0406 19:07:12.837398 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:12.837515 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:12.837612 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:12.837713 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:12.837900 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:12.837959 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:12.838025 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:12.838109 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:12.838163 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:12.867715 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:12.870611 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.870704 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.870767 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:12.908113 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:13.257373 12299 executor.go:111] Tasks: 0 done / 119 total; 48 can run W0406 19:07:13.257609 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:13.257725 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:13.257778 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:13.257923 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:13.257942 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:13.257920 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:13.260481 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:13.264589 12299 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0406 19:07:13.269572 12299 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0406 19:07:13.270063 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.270259 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.270330 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.270544 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.272879 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.277124 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.277272 12299 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0406 19:07:13.277786 12299 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0406 19:07:13.277944 12299 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0406 19:07:13.355456 12299 target.go:112] Terraform output is in /tmp/test535370704/out I0406 19:07:13.361091 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:13.361156 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:13.361208 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:13.361265 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:13.361309 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:13.361351 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:13.361381 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:13.361435 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:13.361465 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:13.388847 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:13.391642 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.391726 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.391780 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.430182 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:13.774095 12299 executor.go:111] Tasks: 0 done / 119 total; 48 can run W0406 19:07:13.774532 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:13.775721 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:13.776726 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:13.776756 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:13.777038 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:13.777622 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:13.777909 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0406 19:07:13.777900 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:13.777942 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:13.778300 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:13.781962 12299 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0406 19:07:13.787394 12299 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0406 19:07:13.787635 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.788107 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.788314 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.788401 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.788997 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.792368 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.792667 12299 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0406 19:07:13.793907 12299 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0406 19:07:13.795113 12299 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0406 19:07:13.799868 12299 target.go:168] Cloudformation output is in /tmp/test1328588748/out --- PASS: TestPrivateCalico (0.98s) === RUN TestPrivateCilium I0406 19:07:13.813958 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:13.814014 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:13.814062 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:13.814112 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:13.814194 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:13.814224 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:13.814254 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:13.814291 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:13.814320 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:13.841913 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:13.844590 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.844671 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.844724 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:13.882754 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:14.040775 12299 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0406 19:07:14.042130 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:14.042436 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:14.042633 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:14.042670 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:14.042754 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:14.043791 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:14.044649 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:14.048547 12299 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0406 19:07:14.051959 12299 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0406 19:07:14.052242 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.052411 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.052642 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.052758 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.053293 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.053924 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.053995 12299 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0406 19:07:14.054264 12299 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0406 19:07:14.054370 12299 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0406 19:07:14.121635 12299 target.go:112] Terraform output is in /tmp/test751681642/out I0406 19:07:14.126656 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:14.126721 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:14.126777 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:14.126840 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:14.126893 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:14.126925 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:14.126958 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:14.127003 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:14.127042 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:14.158705 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:14.161916 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.162015 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.162078 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.201034 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:14.343077 12299 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0406 19:07:14.343431 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:14.343474 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:14.343581 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:14.343407 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:14.343637 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:14.343683 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:14.343699 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:14.343793 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } W0406 19:07:14.344033 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:14.344205 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:14.345857 12299 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0406 19:07:14.350775 12299 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0406 19:07:14.351024 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.351545 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.351998 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.352428 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.353587 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.357068 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.357235 12299 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0406 19:07:14.357585 12299 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0406 19:07:14.357728 12299 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0406 19:07:14.361713 12299 target.go:168] Cloudformation output is in /tmp/test982794613/out --- PASS: TestPrivateCilium (0.56s) === RUN TestPrivateCilium2 I0406 19:07:14.374993 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:14.375052 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:14.375098 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:14.375147 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:14.375255 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:14.375286 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:14.375315 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:14.375349 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:14.375374 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:14.401850 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0406 19:07:14.404530 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.404614 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:14.404662 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.718573 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:15.874432 12299 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0406 19:07:15.874718 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:15.876220 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:15.877660 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:15.877699 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:15.877933 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:15.878665 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:15.879297 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:15.882414 12299 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0406 19:07:15.887932 12299 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0406 19:07:15.888552 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.888849 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.889109 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.892226 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.892352 12299 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0406 19:07:15.892692 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.894407 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:15.894674 12299 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0406 19:07:15.895289 12299 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0406 19:07:15.974523 12299 target.go:112] Terraform output is in /tmp/test3277160381/out I0406 19:07:15.979287 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:15.979352 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:15.979404 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:15.979459 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:15.979502 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:15.979531 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:15.979560 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:15.979595 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:15.979620 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:16.013023 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0406 19:07:16.016463 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.016575 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.016638 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.058559 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:16.211169 12299 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0406 19:07:16.211717 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:16.211775 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:16.211833 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:16.212035 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:16.212105 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:16.212065 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:16.212139 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:16.212232 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:16.212307 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:16.212208 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0406 19:07:16.214287 12299 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0406 19:07:16.219683 12299 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0406 19:07:16.220070 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.220310 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.220468 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.222430 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.222573 12299 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0406 19:07:16.222798 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.223879 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.223994 12299 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0406 19:07:16.224251 12299 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0406 19:07:16.231362 12299 target.go:168] Cloudformation output is in /tmp/test2858111494/out --- PASS: TestPrivateCilium2 (1.87s) === RUN TestPrivateCiliumAdvanced I0406 19:07:16.245398 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:16.245449 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:16.245494 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:16.245548 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:16.245626 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:16.245657 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:16.245685 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:16.245720 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:16.245748 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:16.277395 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:16.280284 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.280362 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.280416 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.318289 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:16.486595 12299 executor.go:111] Tasks: 0 done / 124 total; 54 can run I0406 19:07:16.487159 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:16.487287 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:16.487208 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:16.487355 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:16.487683 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:16.487725 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:16.488950 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:16.495264 12299 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0406 19:07:16.501813 12299 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0406 19:07:16.502028 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.502570 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.502742 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.502962 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.503278 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.505354 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.505488 12299 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0406 19:07:16.506011 12299 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0406 19:07:16.506216 12299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 19:07:16.590234 12299 target.go:112] Terraform output is in /tmp/test2657746498/out I0406 19:07:16.595074 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:16.595142 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:16.595196 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:16.595249 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:16.595295 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:16.595324 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:16.595354 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:16.595391 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:16.595420 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:16.627905 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:16.630849 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.630943 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.631001 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.668996 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:16.831097 12299 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0406 19:07:16.831371 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:16.831835 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:16.831858 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:16.831871 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:16.831968 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:16.832179 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:16.832148 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:16.832337 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:16.832369 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:16.832518 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0406 19:07:16.838460 12299 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0406 19:07:16.845194 12299 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0406 19:07:16.845431 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.845571 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.845722 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.845985 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.846575 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.848319 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.848463 12299 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0406 19:07:16.848919 12299 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0406 19:07:16.849160 12299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 19:07:16.854061 12299 target.go:168] Cloudformation output is in /tmp/test4074862547/out --- PASS: TestPrivateCiliumAdvanced (0.63s) === RUN TestPrivateCanal I0406 19:07:16.871792 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:16.871845 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:16.871891 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:16.871943 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:16.872050 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:16.872082 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:16.872111 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:16.872146 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:16.872174 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:16.898637 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0406 19:07:16.901430 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.901520 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:16.901578 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:17.964903 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:18.370835 12299 executor.go:111] Tasks: 0 done / 119 total; 49 can run I0406 19:07:18.371306 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:18.371342 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:18.371471 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:18.371817 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:18.372534 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:18.372713 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:18.373609 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:18.379386 12299 executor.go:111] Tasks: 49 done / 119 total; 28 can run I0406 19:07:18.385155 12299 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0406 19:07:18.385491 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:18.385743 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:18.385928 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:18.386014 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:18.387162 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:18.389058 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:18.389195 12299 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0406 19:07:18.389595 12299 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0406 19:07:18.389760 12299 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0406 19:07:18.468707 12299 target.go:112] Terraform output is in /tmp/test1582043470/out --- PASS: TestPrivateCanal (1.60s) === RUN TestPrivateKopeio I0406 19:07:18.474730 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:18.474811 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:18.474869 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:18.474922 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:18.475012 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:18.475045 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:18.475077 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:18.475113 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:18.475141 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:18.509878 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:18.512949 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.513047 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.513112 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.551671 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:18.674161 12299 executor.go:111] Tasks: 0 done / 124 total; 48 can run W0406 19:07:18.674436 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:18.674567 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:18.674581 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:18.674591 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:18.674700 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:18.674910 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:18.675161 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:18.680698 12299 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0406 19:07:18.685674 12299 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0406 19:07:18.685929 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.686147 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.686371 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.686575 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.687436 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.688961 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.689090 12299 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0406 19:07:18.689600 12299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 19:07:18.771910 12299 target.go:112] Terraform output is in /tmp/test4166231853/out --- PASS: TestPrivateKopeio (0.30s) === RUN TestUnmanaged I0406 19:07:18.776759 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:18.776814 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:18.776864 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:18.776914 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:18.776958 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:18.776987 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:18.777018 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:18.777048 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:18.777075 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:18.806485 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:18.809548 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.809639 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.809701 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.847807 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:18.961602 12299 executor.go:111] Tasks: 0 done / 106 total; 46 can run I0406 19:07:18.962149 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:18.962194 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:18.962316 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:18.962356 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:18.962415 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:18.965258 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:18.965493 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:18.969853 12299 executor.go:111] Tasks: 46 done / 106 total; 25 can run I0406 19:07:18.974691 12299 executor.go:111] Tasks: 71 done / 106 total; 31 can run I0406 19:07:18.975006 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.975198 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.975320 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.975515 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.976591 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.978250 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:18.978396 12299 executor.go:111] Tasks: 102 done / 106 total; 4 can run I0406 19:07:18.978849 12299 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0406 19:07:19.049259 12299 target.go:112] Terraform output is in /tmp/test2077401657/out --- PASS: TestUnmanaged (0.28s) === RUN TestPrivateSharedSubnet I0406 19:07:19.053793 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:19.053854 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:19.053905 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:19.053959 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:19.054000 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:19.054029 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:19.054057 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:19.054089 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:19.054118 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:19.082162 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:19.085204 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.085294 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.085352 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.124723 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:19.249503 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0406 19:07:19.249771 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:19.249945 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:19.249942 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:19.249992 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:19.250145 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:19.250348 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:19.250510 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:19.252496 12299 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0406 19:07:19.253142 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:19.257624 12299 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0406 19:07:19.257851 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.258017 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.258112 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.258236 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.261534 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.264290 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.264455 12299 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0406 19:07:19.264968 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:19.333726 12299 target.go:112] Terraform output is in /tmp/test3237159724/out --- PASS: TestPrivateSharedSubnet (0.28s) === RUN TestPrivateSharedIP I0406 19:07:19.338283 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:19.338345 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:19.338397 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:19.338451 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:19.338547 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:19.338577 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:19.338610 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:19.338647 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:19.338674 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:19.370117 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:19.373207 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.373310 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.373375 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.411469 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:19.528886 12299 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0406 19:07:19.529557 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:19.529602 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:19.529721 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:19.529638 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:19.529770 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:19.529844 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:19.530017 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:19.532366 12299 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0406 19:07:19.532987 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:19.537409 12299 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0406 19:07:19.538765 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.539072 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.539878 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.540179 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.543344 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.545125 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.545266 12299 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0406 19:07:19.545789 12299 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0406 19:07:19.545956 12299 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0406 19:07:19.618378 12299 target.go:112] Terraform output is in /tmp/test1904477845/out I0406 19:07:19.623005 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:19.623071 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:19.623125 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:19.623178 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:19.623223 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:19.623255 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:19.623286 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:19.623321 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:19.623349 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:19.655848 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:19.658795 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.658892 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.658955 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.697272 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:19.809903 12299 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0406 19:07:19.810258 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:19.810406 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0406 19:07:19.810450 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:19.810424 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:19.810521 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:19.810759 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:19.810790 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:19.810898 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:19.810923 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:19.810924 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:19.814120 12299 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0406 19:07:19.815228 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:19.821272 12299 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0406 19:07:19.821867 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.822107 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.822253 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.822326 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.822921 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.824520 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:19.824674 12299 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0406 19:07:19.825054 12299 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0406 19:07:19.825192 12299 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0406 19:07:19.828999 12299 target.go:168] Cloudformation output is in /tmp/test191733767/out --- PASS: TestPrivateSharedIP (0.50s) === RUN TestPrivateDns1 I0406 19:07:19.841940 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:19.841981 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:19.842015 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:19.842053 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:19.842119 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:19.842142 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:19.842165 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:19.842191 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:19.842213 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:19.877204 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:19.877297 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:19.877360 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.050270 12299 executor.go:111] Tasks: 0 done / 118 total; 47 can run W0406 19:07:20.050822 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:20.050892 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:20.051067 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:20.051331 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:20.057150 12299 executor.go:111] Tasks: 47 done / 118 total; 24 can run I0406 19:07:20.057542 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0406 19:07:20.057597 12299 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0406 19:07:20.057727 12299 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0406 19:07:20.057759 12299 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0406 19:07:20.057894 12299 executor.go:111] Tasks: 71 done / 118 total; 35 can run I0406 19:07:20.058443 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.058659 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.061269 12299 executor.go:111] Tasks: 106 done / 118 total; 5 can run W0406 19:07:20.061463 12299 api.go:53] PageSize not implemented W0406 19:07:20.061519 12299 api.go:53] PageSize not implemented W0406 19:07:20.061549 12299 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0406 19:07:20.061521 12299 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0406 19:07:20.067733 12299 executor.go:111] Tasks: 111 done / 118 total; 5 can run I0406 19:07:20.068064 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.068338 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.069350 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.070660 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.070792 12299 executor.go:111] Tasks: 116 done / 118 total; 2 can run I0406 19:07:20.071226 12299 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0406 19:07:20.161039 12299 target.go:112] Terraform output is in /tmp/test1056066030/out --- PASS: TestPrivateDns1 (0.32s) === RUN TestPrivateDns2 I0406 19:07:20.166216 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:20.166275 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:20.166326 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:20.166375 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:20.166417 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:20.166446 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:20.166477 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:20.166513 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:20.166540 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:20.197824 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.197916 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.197981 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.342396 12299 executor.go:111] Tasks: 0 done / 114 total; 45 can run W0406 19:07:20.343132 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:20.343151 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:20.343379 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:20.343537 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:20.345190 12299 executor.go:111] Tasks: 45 done / 114 total; 22 can run I0406 19:07:20.345463 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:20.345792 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0406 19:07:20.345846 12299 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0406 19:07:20.345941 12299 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0406 19:07:20.345960 12299 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0406 19:07:20.346044 12299 executor.go:111] Tasks: 67 done / 114 total; 35 can run I0406 19:07:20.346466 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.346701 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.349500 12299 executor.go:111] Tasks: 102 done / 114 total; 5 can run W0406 19:07:20.349579 12299 api.go:53] PageSize not implemented W0406 19:07:20.349603 12299 api.go:53] PageSize not implemented W0406 19:07:20.349601 12299 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0406 19:07:20.349635 12299 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0406 19:07:20.357872 12299 executor.go:111] Tasks: 107 done / 114 total; 5 can run I0406 19:07:20.358145 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.358362 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.359311 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.360619 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.360742 12299 executor.go:111] Tasks: 112 done / 114 total; 2 can run I0406 19:07:20.361030 12299 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0406 19:07:20.433286 12299 target.go:112] Terraform output is in /tmp/test1706339639/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I0406 19:07:20.436095 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:20.436137 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:20.436169 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:20.436204 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:20.436234 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:20.436253 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:20.436274 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:20.436297 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:20.436316 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:20.464417 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0406 19:07:20.467913 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:20.468011 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.529126 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:21.651225 12299 executor.go:111] Tasks: 0 done / 91 total; 47 can run I0406 19:07:21.652162 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:21.652205 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:21.652239 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:21.652314 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:21.652244 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:21.652374 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:21.655227 12299 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0406 19:07:21.663056 12299 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0406 19:07:21.663258 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.663400 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.664372 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.665770 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.665896 12299 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0406 19:07:21.666211 12299 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0406 19:07:21.722535 12299 target.go:112] Terraform output is in /tmp/test95556882/out --- PASS: TestDiscoveryFeatureGate (1.29s) === RUN TestVFSServiceAccountIssuerDiscovery I0406 19:07:21.727232 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:21.727302 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:21.727353 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:21.727405 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:21.727447 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:21.727475 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:21.727505 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:21.727546 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:21.727574 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:21.756030 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:21.759218 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.759384 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.797121 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:21.912953 12299 executor.go:111] Tasks: 0 done / 89 total; 46 can run I0406 19:07:21.913411 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:21.913586 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:21.913617 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:21.913699 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:21.913799 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:21.914136 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:21.916768 12299 executor.go:111] Tasks: 46 done / 89 total; 20 can run I0406 19:07:21.928176 12299 executor.go:111] Tasks: 66 done / 89 total; 21 can run I0406 19:07:21.928607 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.928993 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.930141 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.931137 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:21.931252 12299 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0406 19:07:21.931666 12299 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0406 19:07:21.988446 12299 target.go:112] Terraform output is in /tmp/test2319678080/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.27s) === RUN TestAWSLBController I0406 19:07:21.992935 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:21.993002 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:21.993056 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:21.993112 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:21.993215 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:21.993246 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:21.993278 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:21.993316 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:21.993346 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:22.018638 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:22.021694 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:22.021792 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:22.059137 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:23.411983 12299 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0406 19:07:23.412640 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:23.412720 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:23.412715 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:23.412741 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:23.412803 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:23.414663 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:23.433274 12299 executor.go:111] Tasks: 50 done / 95 total; 22 can run I0406 19:07:23.438955 12299 executor.go:111] Tasks: 72 done / 95 total; 21 can run I0406 19:07:23.439373 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:23.439602 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:23.440749 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:23.441861 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:23.441989 12299 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0406 19:07:23.442309 12299 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0406 19:07:23.501358 12299 target.go:112] Terraform output is in /tmp/test1822166027/out --- PASS: TestAWSLBController (1.53s) === RUN TestManyAddons I0406 19:07:23.521564 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:23.521632 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:23.521757 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:23.521808 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:23.521851 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:23.521880 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:23.521910 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:23.521944 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:23.521972 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:23.549166 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:23.552358 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:23.552443 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:23.593355 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:25.117291 12299 executor.go:111] Tasks: 0 done / 94 total; 51 can run I0406 19:07:25.117570 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:25.118031 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:25.118056 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:25.118375 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:25.118425 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:25.118592 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:25.135025 12299 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0406 19:07:25.144093 12299 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0406 19:07:25.144406 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:25.144595 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:25.145550 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:25.147281 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:25.147397 12299 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0406 19:07:25.147703 12299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0406 19:07:25.200962 12299 target.go:112] Terraform output is in /tmp/test3433070839/out --- PASS: TestManyAddons (1.70s) === RUN TestManyAddonsCCMIRSA I0406 19:07:25.222269 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:25.222333 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:25.222446 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:25.222501 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:25.222549 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:25.222578 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:25.222612 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:25.222648 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:25.222677 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:25.250914 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:25.254498 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:25.254617 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:25.294532 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:26.899929 12299 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0406 19:07:26.900717 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:26.900762 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:26.900861 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:26.902309 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:26.903269 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:26.903886 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:26.924111 12299 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0406 19:07:26.929888 12299 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0406 19:07:26.930194 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:26.930338 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:26.931563 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:26.932927 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:26.933063 12299 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0406 19:07:26.933484 12299 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0406 19:07:27.003339 12299 target.go:112] Terraform output is in /tmp/test2919145766/out --- PASS: TestManyAddonsCCMIRSA (1.80s) === RUN TestCCM I0406 19:07:27.026737 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:27.026815 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:27.026893 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:27.026982 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:27.027109 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:27.027171 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:27.027229 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:27.027289 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:27.027336 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:27.058119 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:27.061597 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:27.061703 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:27.100232 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:28.673227 12299 executor.go:111] Tasks: 0 done / 95 total; 52 can run W0406 19:07:28.673533 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:28.673797 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:28.673816 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:28.673890 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:28.674005 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:28.674000 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:28.689326 12299 executor.go:111] Tasks: 52 done / 95 total; 19 can run I0406 19:07:28.698432 12299 executor.go:111] Tasks: 71 done / 95 total; 22 can run I0406 19:07:28.698792 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.699088 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.700391 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.701519 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.701637 12299 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0406 19:07:28.701936 12299 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0406 19:07:28.757184 12299 target.go:112] Terraform output is in /tmp/test330411441/out --- PASS: TestCCM (1.75s) === RUN TestExternalDNS I0406 19:07:28.780003 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:28.780067 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:28.780118 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:28.780169 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:28.780252 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:28.780281 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:28.780310 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:28.780347 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:28.780375 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:28.804360 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:28.807117 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.807207 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.846460 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:28.962196 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0406 19:07:28.962696 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:28.962769 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:28.963226 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:28.963248 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:28.963265 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:28.963390 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:28.965517 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:28.974224 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:28.974571 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.974849 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.976305 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.977298 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:28.977412 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:28.977740 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:29.031600 12299 target.go:112] Terraform output is in /tmp/test304670824/out I0406 19:07:29.035794 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:29.035855 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:29.035909 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:29.035958 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:29.035999 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:29.036027 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:29.036056 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:29.036089 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:29.036115 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:29.059920 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:29.062718 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.062809 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.101332 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:29.215861 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:29.216113 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:29.216136 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:29.216223 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:29.216357 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:29.216774 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:29.216947 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:29.217097 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:29.217116 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:29.217208 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:29.219495 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:29.229670 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:29.229970 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.230133 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.231229 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.232197 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.232331 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:29.232657 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:29.236048 12299 target.go:168] Cloudformation output is in /tmp/test4200060732/out --- PASS: TestExternalDNS (0.47s) === RUN TestExternalDNSIRSA I0406 19:07:29.246560 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:29.246615 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:29.246659 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:29.246710 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:29.246754 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:29.246782 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:29.246814 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:29.246849 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:29.246878 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:29.275273 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:29.278227 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.278335 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.315637 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:29.444242 12299 executor.go:111] Tasks: 0 done / 91 total; 47 can run W0406 19:07:29.444913 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:29.445187 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:29.445235 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:29.445237 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:29.445389 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:29.445461 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:29.447318 12299 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0406 19:07:29.457167 12299 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0406 19:07:29.457409 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.457759 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.459203 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.460679 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.460806 12299 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0406 19:07:29.461175 12299 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0406 19:07:29.521439 12299 target.go:112] Terraform output is in /tmp/test2668870841/out --- PASS: TestExternalDNSIRSA (0.28s) === RUN TestSharedSubnet I0406 19:07:29.526197 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:29.526252 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:29.526300 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:29.526351 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:29.526392 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:29.526424 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:29.526455 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:29.526487 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:29.526514 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:29.550409 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:29.553144 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.553236 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.591190 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:29.704298 12299 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0406 19:07:29.704685 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:29.707272 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:29.707797 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:29.707844 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:29.707884 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:29.708025 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:29.712487 12299 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0406 19:07:29.713015 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:29.718008 12299 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0406 19:07:29.718542 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.718758 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.719613 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.721566 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.721689 12299 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0406 19:07:29.722037 12299 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0406 19:07:29.769534 12299 target.go:112] Terraform output is in /tmp/test2889788079/out --- PASS: TestSharedSubnet (0.25s) === RUN TestSharedVPC I0406 19:07:29.773824 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:29.773879 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:29.773931 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:29.773982 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:29.774026 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:29.774055 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:29.774085 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:29.774118 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:29.774146 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:29.802208 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:29.804966 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.805065 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.842185 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:29.955488 12299 executor.go:111] Tasks: 0 done / 83 total; 43 can run W0406 19:07:29.955766 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:29.956045 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:29.956262 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:29.956341 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:29.956312 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:29.956451 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:29.958463 12299 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0406 19:07:29.958906 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:29.967543 12299 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0406 19:07:29.967973 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.968141 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.969315 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.970274 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:29.970391 12299 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0406 19:07:29.970880 12299 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0406 19:07:30.021201 12299 target.go:112] Terraform output is in /tmp/test531338596/out --- PASS: TestSharedVPC (0.25s) === RUN TestExistingIAM I0406 19:07:30.028184 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:30.028245 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:30.028301 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:30.028356 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:30.028400 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:30.028429 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:30.028460 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:30.028492 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:30.028522 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:30.061541 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:30.064671 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.064847 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.064923 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.064986 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.103174 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:30.213380 12299 executor.go:111] Tasks: 0 done / 94 total; 48 can run I0406 19:07:30.213892 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:30.213928 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:30.214025 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:30.214080 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:30.214143 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:30.214154 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:30.214334 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:30.214481 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:30.216878 12299 executor.go:111] Tasks: 48 done / 94 total; 15 can run I0406 19:07:30.225898 12299 executor.go:111] Tasks: 63 done / 94 total; 27 can run I0406 19:07:30.226216 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.226407 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.226588 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.226871 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.227373 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.229012 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.230306 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.230609 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:30.230739 12299 executor.go:111] Tasks: 90 done / 94 total; 4 can run I0406 19:07:30.231195 12299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0406 19:07:30.307906 12299 target.go:112] Terraform output is in /tmp/test1665167686/out --- PASS: TestExistingIAM (0.28s) === RUN TestPhaseNetwork I0406 19:07:30.312294 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:30.312359 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:30.312413 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:30.312469 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:30.312515 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:30.312543 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:30.312573 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:30.312610 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:30.312640 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:30.340737 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:30.343577 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.343747 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.343807 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.381750 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:30.496181 12299 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0406 19:07:30.496460 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:30.496707 12299 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0406 19:07:30.497184 12299 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0406 19:07:30.497677 12299 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0406 19:07:30.497962 12299 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0406 19:07:30.498201 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0406 19:07:30.507067 12299 target.go:112] Terraform output is in /tmp/test179412850/out --- PASS: TestPhaseNetwork (0.20s) === RUN TestExternalLoadBalancer I0406 19:07:30.508090 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:30.508150 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:30.508203 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:30.508257 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:30.508356 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:30.508386 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:30.508418 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:30.508456 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:30.508486 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:30.533538 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:30.536427 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.536534 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.573871 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:30.684878 12299 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0406 19:07:30.685764 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:30.685764 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:30.686107 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:30.686161 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:30.686345 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:30.686328 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:30.691065 12299 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0406 19:07:30.697604 12299 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0406 19:07:30.698242 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.698465 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.699557 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.700902 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.701020 12299 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0406 19:07:30.701350 12299 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0406 19:07:30.756054 12299 target.go:112] Terraform output is in /tmp/test1412575564/out I0406 19:07:30.760229 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:30.760284 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:30.760332 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:30.760380 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:30.760422 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:30.760450 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:30.760480 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:30.760512 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:30.760538 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:30.785378 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:30.788157 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.788246 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.825285 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:30.936664 12299 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0406 19:07:30.937326 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:30.937526 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:30.938254 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:30.937557 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:30.938342 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:30.938329 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:30.938753 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:30.939655 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:30.941424 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:30.944702 12299 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0406 19:07:30.950495 12299 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0406 19:07:30.950823 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.951122 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.952258 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.953728 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:30.953888 12299 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0406 19:07:30.954212 12299 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0406 19:07:30.957207 12299 target.go:168] Cloudformation output is in /tmp/test1178453837/out --- PASS: TestExternalLoadBalancer (0.46s) === RUN TestPhaseIAM integration_test.go:633: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:641: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0406 19:07:30.967679 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:30.967734 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:30.967779 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:30.967833 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:30.967919 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:30.967951 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:30.967986 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:30.968023 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:30.968053 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:31.005758 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:31.008862 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.008968 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.009029 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.009087 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.009143 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.009194 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.009246 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.047415 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:31.169726 12299 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0406 19:07:31.170067 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:31.170118 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:31.170211 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:31.170404 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:31.170519 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:31.170659 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:31.170691 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:31.170847 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:31.172818 12299 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0406 19:07:31.182505 12299 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0406 19:07:31.183010 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.183238 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.183516 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.183791 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.184347 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.185390 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.185817 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.186138 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.186268 12299 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0406 19:07:31.186582 12299 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0406 19:07:31.268466 12299 target.go:112] Terraform output is in /tmp/test1371919766/out I0406 19:07:31.273332 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:31.273395 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:31.273449 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:31.273503 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:31.273545 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:31.273575 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:31.273605 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:31.273641 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:31.273669 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:31.308565 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:31.311686 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.311787 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.311847 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.311906 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.311958 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.312009 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.312055 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.350584 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:31.465643 12299 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0406 19:07:31.466242 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:31.466261 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:31.466407 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0406 19:07:31.466455 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:31.466429 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:31.466529 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0406 19:07:31.466690 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:31.466292 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:31.466300 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:31.466733 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:31.466896 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:31.468437 12299 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0406 19:07:31.477957 12299 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0406 19:07:31.478397 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.478607 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.478827 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.479109 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.480091 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.480998 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.481274 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.481568 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.481707 12299 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0406 19:07:31.482074 12299 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0406 19:07:31.487484 12299 target.go:168] Cloudformation output is in /tmp/test2449418149/out --- PASS: TestMixedInstancesASG (0.54s) === RUN TestMixedInstancesSpotASG I0406 19:07:31.507731 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:31.507784 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:31.507829 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:31.507879 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:31.507968 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:31.507998 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:31.508027 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:31.508060 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:31.508107 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:31.539776 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:31.542414 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.542492 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.542548 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.542600 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.542645 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.542688 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.542730 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.580784 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:31.696113 12299 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0406 19:07:31.696533 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:31.696909 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 19:07:31.696994 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:31.696938 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:31.697044 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:31.697057 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:31.697139 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:31.697230 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:31.699593 12299 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0406 19:07:31.709729 12299 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0406 19:07:31.710004 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.710197 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.710407 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.710659 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.711791 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.712573 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.712815 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.713106 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.713224 12299 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0406 19:07:31.713737 12299 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0406 19:07:31.794521 12299 target.go:112] Terraform output is in /tmp/test2095358197/out I0406 19:07:31.799478 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:31.799538 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:31.799594 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:31.799645 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:31.799690 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:31.799720 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:31.799750 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:31.799786 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:31.799814 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:31.832179 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:31.835042 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.835121 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.835175 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.835229 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.835277 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.835318 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.835361 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:31.873414 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:31.989181 12299 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0406 19:07:31.989703 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:31.989930 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:31.989996 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:31.989974 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:31.990094 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:31.990142 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:31.990195 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:31.990223 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:31.990355 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:31.990053 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:31.990390 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0406 19:07:31.992615 12299 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0406 19:07:32.001389 12299 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0406 19:07:32.001731 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.001881 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.002099 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.002342 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.002902 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.004265 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.005906 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.006062 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.006209 12299 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0406 19:07:32.006603 12299 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0406 19:07:32.013030 12299 target.go:168] Cloudformation output is in /tmp/test3278505301/out --- PASS: TestMixedInstancesSpotASG (0.52s) === RUN TestContainerd I0406 19:07:32.031757 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:32.031816 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:32.031863 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:32.031919 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:32.032002 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:32.032032 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:32.032063 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:32.032102 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:32.032132 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:32.060540 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:32.063409 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.063507 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.100936 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:32.213365 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:32.213879 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:32.213913 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.213921 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:32.214040 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:32.214129 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:32.214199 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:32.214224 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:32.214235 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:32.214301 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0406 19:07:32.216554 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:32.221574 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:32.222145 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.222871 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.225246 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.228476 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.228657 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:32.228965 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:32.232209 12299 target.go:168] Cloudformation output is in /tmp/test1471990423/out --- PASS: TestContainerd (0.21s) === RUN TestContainerdCustom I0406 19:07:32.243444 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:32.243506 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:32.243586 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:32.243669 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:32.243767 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:32.243826 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:32.243884 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:32.243940 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:32.243988 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:32.269074 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:32.271954 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.272057 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.309096 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:32.423840 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:32.424499 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:32.424614 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.424546 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 19:07:32.424671 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.424671 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:32.424688 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:32.424871 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:32.425296 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:32.426260 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" },{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" }] }] } I0406 19:07:32.431898 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:32.436971 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:32.437292 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.437469 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.438522 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.439923 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.440066 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:32.440314 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:32.443563 12299 target.go:168] Cloudformation output is in /tmp/test1137014565/out --- PASS: TestContainerdCustom (0.21s) === RUN TestDockerCustom I0406 19:07:32.454083 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:32.454137 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:32.454191 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:32.454245 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:32.454322 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:32.454351 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:32.454381 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:32.454413 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:32.454440 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:32.484808 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:32.487840 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.487935 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.525165 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:32.639634 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:32.639901 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:32.640051 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:32.640068 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:32.640206 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.640160 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I0406 19:07:32.640305 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:32.640353 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.640347 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:32.640436 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:32.647228 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:32.652971 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:32.653291 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.653493 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.654662 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.656143 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:32.656278 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:32.656533 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:32.659571 12299 target.go:168] Cloudformation output is in /tmp/test2219169379/out --- PASS: TestDockerCustom (0.22s) === RUN TestAPIServerNodes I0406 19:07:32.669511 12299 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0406 19:07:32.669713 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:32.669767 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:32.669850 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:32.669901 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:32.669943 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:32.669972 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:32.670001 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:32.670034 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:32.670063 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:32.700347 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0406 19:07:32.703251 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.703348 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.703412 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.745487 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:32.915589 12299 executor.go:111] Tasks: 0 done / 98 total; 48 can run W0406 19:07:32.917128 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.917442 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0406 19:07:32.917454 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.917506 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:32.917490 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:32.917642 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0406 19:07:32.917762 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:32.917797 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:32.917951 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0406 19:07:32.918627 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:32.922623 12299 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0406 19:07:32.928724 12299 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0406 19:07:32.928986 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.929262 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.929592 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.930154 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.930604 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.932346 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.932495 12299 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0406 19:07:32.932841 12299 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0406 19:07:32.936499 12299 target.go:168] Cloudformation output is in /tmp/test2609373447/out I0406 19:07:32.952494 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:32.952552 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:32.952597 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:32.952642 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:32.952735 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:32.952765 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:32.952793 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:32.952826 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:32.952852 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:32.980281 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0406 19:07:32.983193 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.983281 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:32.983342 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.022253 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:33.194588 12299 executor.go:111] Tasks: 0 done / 98 total; 48 can run I0406 19:07:33.195127 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:33.195170 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 19:07:33.195254 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:33.195265 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:33.195555 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:33.195677 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:33.195856 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:33.197389 12299 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0406 19:07:33.209074 12299 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0406 19:07:33.209486 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.209695 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.209949 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.210683 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.210980 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.212980 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.213101 12299 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0406 19:07:33.213499 12299 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0406 19:07:33.280101 12299 target.go:112] Terraform output is in /tmp/test727593203/out I0406 19:07:33.284668 12299 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.62s) === RUN TestNTHQueueProcessor I0406 19:07:33.285044 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:33.285103 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:33.285154 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:33.285211 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:33.285308 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:33.285337 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:33.285370 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:33.285407 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:33.285436 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:33.310046 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0406 19:07:33.313038 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:33.313135 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.588024 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:34.719241 12299 executor.go:111] Tasks: 0 done / 98 total; 46 can run W0406 19:07:34.719637 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:34.719810 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:34.719846 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:34.720828 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 19:07:34.723529 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:34.723524 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:34.727406 12299 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0406 19:07:34.732566 12299 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0406 19:07:34.733185 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.733382 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.734292 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.735862 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.735985 12299 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0406 19:07:34.736316 12299 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0406 19:07:34.736529 12299 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0406 19:07:34.799779 12299 target.go:112] Terraform output is in /tmp/test3487204816/out I0406 19:07:34.804601 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:34.804660 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:34.804712 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:34.804790 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:34.804840 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:34.804871 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:34.804905 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:34.804941 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:34.804969 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:34.829547 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0406 19:07:34.832442 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.832539 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:34.875071 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0406 19:07:35.003874 12299 executor.go:111] Tasks: 0 done / 98 total; 46 can run W0406 19:07:35.004120 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0406 19:07:35.004529 12299 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0406 19:07:35.005479 12299 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0406 19:07:35.006912 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:35.007075 12299 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0406 19:07:35.007097 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:35.007232 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:35.010076 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:35.010201 12299 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0406 19:07:35.012630 12299 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0406 19:07:35.018418 12299 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0406 19:07:35.018872 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:35.019157 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:35.020252 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:35.021328 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:35.021500 12299 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0406 19:07:35.021809 12299 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0406 19:07:35.022110 12299 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0406 19:07:35.025842 12299 target.go:168] Cloudformation output is in /tmp/test1980475988/out --- PASS: TestNTHQueueProcessor (1.76s) === RUN TestCustomIRSA I0406 19:07:35.040511 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:35.040566 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:35.040614 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:35.040664 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:35.040759 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:35.040790 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:35.040821 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:35.040856 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:35.040885 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:35.069917 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0406 19:07:35.073139 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:35.073235 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:35.111275 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:36.444257 12299 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0406 19:07:36.444575 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:36.445294 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:36.445363 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:36.445396 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:36.445459 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:36.445537 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:36.460433 12299 executor.go:111] Tasks: 51 done / 99 total; 25 can run I0406 19:07:36.469759 12299 executor.go:111] Tasks: 76 done / 99 total; 21 can run I0406 19:07:36.470305 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.470595 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.471422 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.472899 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.473025 12299 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0406 19:07:36.473359 12299 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0406 19:07:36.533786 12299 target.go:112] Terraform output is in /tmp/test1865835834/out --- PASS: TestCustomIRSA (1.51s) === RUN TestCustomIRSA119 I0406 19:07:36.554980 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:36.555049 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:36.555102 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:36.555156 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:36.555204 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:36.555234 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:36.555268 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:36.555306 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:36.555334 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:36.583230 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0406 19:07:36.586707 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.586803 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.627713 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:36.737776 12299 executor.go:111] Tasks: 0 done / 97 total; 49 can run I0406 19:07:36.738282 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:36.738323 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:36.738417 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:36.738465 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:36.738604 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 19:07:36.738717 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:36.746042 12299 executor.go:111] Tasks: 49 done / 97 total; 25 can run I0406 19:07:36.751397 12299 executor.go:111] Tasks: 74 done / 97 total; 21 can run I0406 19:07:36.751860 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.752069 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.753155 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.754898 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.755027 12299 executor.go:111] Tasks: 95 done / 97 total; 2 can run I0406 19:07:36.755322 12299 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0406 19:07:36.816761 12299 target.go:112] Terraform output is in /tmp/test115453824/out --- PASS: TestCustomIRSA119 (0.27s) === RUN TestClusterNameDigit I0406 19:07:36.821849 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:36.821906 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:36.821956 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:36.822009 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:36.822052 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:36.822081 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:36.822110 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:36.822142 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:36.822170 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:36.849163 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0406 19:07:36.852233 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.852327 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:36.890133 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:37.003004 12299 executor.go:111] Tasks: 0 done / 94 total; 48 can run W0406 19:07:37.003467 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:37.003817 12299 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 19:07:37.003848 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 19:07:37.003919 12299 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 19:07:37.003944 12299 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 19:07:37.003789 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:37.005958 12299 executor.go:111] Tasks: 48 done / 94 total; 23 can run I0406 19:07:37.017454 12299 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0406 19:07:37.018050 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.018290 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.019465 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.020852 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.020972 12299 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0406 19:07:37.021345 12299 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0406 19:07:37.079171 12299 target.go:112] Terraform output is in /tmp/test766351253/out --- PASS: TestClusterNameDigit (0.26s) === RUN TestLifecycleMinimalAWS I0406 19:07:37.083943 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:37.084007 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:37.084058 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:37.084114 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:37.084159 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:37.084187 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:37.084221 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:37.084258 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:37.084287 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:37.084353 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0406 19:07:37.108045 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:37.110827 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.110921 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 19:07:37.154170 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:37.264169 12299 iamrole.go:91] ListRoles: { } I0406 19:07:37.265765 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:37.266257 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0406 19:07:37.266436 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0406 19:07:37.266251 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.266839 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:37.266707 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.266746 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0406 19:07:37.269302 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0406 19:07:37.269471 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0406 19:07:37.269372 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.269401 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.269729 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:37.269853 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] }] } I0406 19:07:37.269857 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0406 19:07:37.270027 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.270303 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:37.270393 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:37.270571 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:37.271793 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:37.272599 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:37.272794 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:37.272858 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:37.273048 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:37.273133 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:37.273190 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:37.273235 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0406 19:07:37.273442 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:37.273581 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:37.273757 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:37.273702 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0406 19:07:37.273821 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:37.273909 12299 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0406 19:07:37.273986 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 19:07:37.274098 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0406 19:07:37.273916 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0406 19:07:37.274212 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.274420 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 19:07:37.274435 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0406 19:07:37.274485 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 19:07:37.274617 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:37.274713 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0406 19:07:37.274854 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 19:07:37.274928 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.275150 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.275248 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:37.275290 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:37.275382 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:37.275436 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:37.275489 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:37.275538 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:37.275584 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:37.275722 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:37.275790 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:37.275845 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:37.275920 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:37.275987 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I0406 19:07:37.276418 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0406 19:07:37.278990 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:37.279237 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.279448 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.279677 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.279957 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.280230 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.280498 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.280640 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.280763 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.280988 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:37.281038 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 19:07:37.281115 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.281523 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.281740 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:37.281846 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.282025 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.282118 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.282264 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.282399 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.282588 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.282712 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.282846 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:37.282902 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.283088 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 19:07:37.283334 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.283524 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.284758 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:37.285074 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:37.285151 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:37.285190 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:37.285260 12299 dns.go:238] Pre-creating DNS records I0406 19:07:37.285289 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:37.290832 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:37.293569 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.293659 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 19:07:37.336297 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:37.447439 12299 iamrole.go:91] ListRoles: { } I0406 19:07:37.449125 12299 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0406 19:07:37.449496 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:37.449451 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.449713 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.449822 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.450083 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.450265 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:37.450324 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:37.450423 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:37.450488 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:37.450577 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:37.451870 12299 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0406 19:07:37.452024 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.452193 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0406 19:07:37.452221 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:37.452269 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0406 19:07:37.452373 12299 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0406 19:07:37.452431 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:37.452487 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0406 19:07:37.454211 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:37.454291 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.454525 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:37.454649 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:37.454796 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.461350 12299 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0406 19:07:37.461530 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.461753 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.461996 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.462129 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.462306 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.462509 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.462710 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:37.464505 12299 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 19:07:37.464695 12299 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 19:07:37.468096 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:37.468162 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:37.468197 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:37.468229 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:37.468267 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:37.468300 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:37.468331 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:37.468364 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:37.468398 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:37.468434 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:37.468475 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:37.468516 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:37.468546 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:37.468577 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:37.468610 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:37.468642 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:37.468673 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:37.468707 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:37.468753 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:37.468786 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:37.468816 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:37.468847 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:37.468883 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:37.468919 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:37.468975 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:37.470715 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:37.470742 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:37.470782 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.470833 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0406 19:07:37.470890 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.470952 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.471000 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.471054 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.471100 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.471139 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0406 19:07:37.471184 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.471233 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 19:07:37.471273 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0406 19:07:37.471316 12299 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0406 19:07:37.471347 12299 api.go:53] PageSize not implemented I0406 19:07:37.471368 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:37.471417 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:37.471440 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:37.471472 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:37.471492 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:37.471545 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:37.471574 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:37.471626 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:37.471668 12299 iamrole.go:91] ListRoles: { } I0406 19:07:37.471732 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:37.471777 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:37.471810 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:37.472546 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 19:07:37.472699 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } autoscaling-config:lt-2 ok autoscaling-group:nodes.minimal.example.com ok I0406 19:07:37.472532 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:37.472652 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0406 19:07:37.472852 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0406 19:07:37.472798 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } iam-instance-profile:masters.minimal.example.com ok volume:vol-1 ok autoscaling-config:lt-1 ok I0406 19:07:37.472924 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 19:07:37.473006 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok internet-gateway:igw-2 ok I0406 19:07:37.473018 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:37.473170 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0406 19:07:37.473360 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:37.473385 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:37.473442 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 19:07:37.473502 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:37.473557 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 19:07:37.473747 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:37.473768 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:37.473811 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 19:07:37.473933 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:37.473984 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0406 19:07:37.474079 12299 iamrole.go:182] ListAttachedRolePolicies: masters.minimal.example.com I0406 19:07:37.474095 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0406 19:07:37.474136 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0406 19:07:37.474185 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0406 19:07:37.474231 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal.example.com I0406 19:07:37.474243 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 19:07:37.474273 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0406 19:07:37.474347 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:37.474440 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:37.474527 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:37.474850 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.39s) === RUN TestLifecycleMinimalOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0406 19:07:37.485530 12299 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:37.549267 12299 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0406 19:07:37.712160 12299 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0406 19:07:37.722281 12299 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0406 19:07:37.770879 12299 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0406 19:07:37.774285 12299 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0406 19:07:37.777426 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:37.777447 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:37.777666 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0406 19:07:37.786666 12299 executor.go:111] Tasks: 97 done / 97 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:37.832947 12299 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0406 19:07:37.997915 12299 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0406 19:07:38.001852 12299 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0406 19:07:38.014849 12299 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0406 19:07:38.017179 12299 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0406 19:07:38.020277 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:38.020698 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:38.021034 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:38.021577 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:38.021937 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:38.022416 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0406 19:07:38.024456 12299 executor.go:111] Tasks: 97 done / 97 total; 0 can run ServerGroup:b7e0ab26-80de-4799-affa-b4d9cb3cde2d ok SecurityGroup:b6748ac1-9463-48ab-9543-d0281a847c54 ok Port:1edff422-8bf8-4333-ae59-cf844988ebfb ok Instance:76665401-4712-484f-881b-c8a0efa0e858 ok Subnet:6675883f-114d-4f89-b712-f98bd44127bc ok ServerGroup:6c9acb33-316a-45a7-8503-9655dee9852f ok SecurityGroup:128b7049-7a0b-4e09-804a-ca90231450a5 ok Port:3fceadcc-dbec-4b4d-9993-ebe6bea47258 ok Port:0caf1d99-1f96-45df-bdca-3a3ca290fcc9 ok Instance:48049ba1-23a6-4faa-9c2c-da44a6c219a6 ok Network:0c2f55da-a348-496c-a988-121b378cbfb6 ok Instance:5c33b1ea-2c4a-46df-88db-8c7e65ec8c09 ok SecurityGroup:5fbd13f9-2107-4aa4-9e74-d04ad8afdcd1 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok --- PASS: TestLifecycleMinimalOpenstack (1.67s) === RUN TestLifecycleMinimalGCE I0406 19:07:39.145767 12299 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true lifecycle_integration_test.go:488: running lifecycle test for cluster minimal-gce.example.com I0406 19:07:39.147446 12299 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0406 19:07:39.399968 12299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 19:07:39.400041 12299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 19:07:39.400131 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0406 19:07:39.400163 12299 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0406 19:07:39.400179 12299 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0406 19:07:39.400193 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:39.400247 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:39.400266 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:39.401554 12299 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0406 19:07:39.408996 12299 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0406 19:07:39.414380 12299 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0406 19:07:39.419356 12299 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0406 19:07:39.419592 12299 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0406 19:07:39.419693 12299 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0406 19:07:39.645688 12299 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 19:07:39.645779 12299 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 19:07:39.645892 12299 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0406 19:07:39.645936 12299 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0406 19:07:39.645960 12299 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0406 19:07:39.645979 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:39.646087 12299 autoscalinggroup.go:160] VMs using Service Account: default I0406 19:07:39.646113 12299 autoscalinggroup.go:168] gsa: default I0406 19:07:39.647665 12299 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0406 19:07:39.650872 12299 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0406 19:07:39.656070 12299 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0406 19:07:39.660758 12299 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0406 19:07:39.660934 12299 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0406 19:07:39.662687 12299 gce.go:93] Scanning zones: [us-test1-a] InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1649272059 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1649272059 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.52s) === RUN TestLifecycleFloatingIPOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0406 19:07:39.672237 12299 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:39.744464 12299 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0406 19:07:39.909130 12299 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0406 19:07:39.918496 12299 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0406 19:07:39.986842 12299 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0406 19:07:39.990437 12299 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0406 19:07:39.993309 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:39.993314 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:39.993463 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0406 19:07:39.998326 12299 executor.go:111] Tasks: 99 done / 99 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:40.045104 12299 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0406 19:07:40.202099 12299 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0406 19:07:40.211149 12299 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0406 19:07:40.223730 12299 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0406 19:07:40.226263 12299 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0406 19:07:40.229281 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:40.229675 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:40.230386 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:40.230659 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:40.231339 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0406 19:07:40.231478 12299 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0406 19:07:40.233670 12299 executor.go:111] Tasks: 99 done / 99 total; 0 can run Router-IF:b2e846e3-011f-4405-ab33-ef4e990d76ad ok Port:48d9c1f2-f401-4bd1-a1a0-f4a6a3f346a9 ok Subnet:b2e846e3-011f-4405-ab33-ef4e990d76ad ok Router:051be894-e31b-4792-a3f9-c00c105ff461 ok Instance:2ba8e4f3-1b70-438d-a7f6-53b7e0938dd5 ok Port:03b4605c-11a1-4fb4-a4b9-6e97171a5e3f ok Port:b6ee921d-d79d-4b03-9863-02b33aec914c ok SecurityGroup:5c20ca5f-6d78-47b6-9a3c-9b234e49f099 ok Instance:10809130-cff6-4643-a334-a5c7eac411cb ok SecurityGroup:fe3eeb18-2ce2-4bbc-9839-8757dc3b72d8 ok ServerGroup:13da4dcf-3bdc-4812-b98e-9754c2655cb7 ok Port:9f6a8e64-14bf-4d9d-8c0b-2b1dfaa4c87a ok ServerGroup:8ac59597-0e8b-45b7-9740-64faab5f4027 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:c9d5e556-e031-40ef-b459-413ca54215a1 ok Instance:d9d03aca-0258-49de-accd-4138c8a9592f ok Network:9e930407-0ec6-44dd-9915-6b71c1862519 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.68s) === RUN TestLifecyclePrivateCalico I0406 19:07:41.349400 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:41.349457 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:41.349502 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:41.349552 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:41.349595 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:41.349623 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:41.349657 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:41.349688 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:41.349716 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:41.349779 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0406 19:07:41.377907 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:41.385187 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.385314 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.385379 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.423502 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:41.763000 12299 iamrole.go:91] ListRoles: { } I0406 19:07:41.764822 12299 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0406 19:07:41.765090 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.765152 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" }] } I0406 19:07:41.765271 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.765354 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:41.765386 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:41.765414 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:41.765450 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:41.765370 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:41.765481 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.766318 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 19:07:41.767511 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0406 19:07:41.767647 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 19:07:41.767950 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0406 19:07:41.768360 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 19:07:41.765610 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 19:07:41.768689 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:41.768663 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0406 19:07:41.768777 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:41.768793 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 19:07:41.768854 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:41.768948 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0406 19:07:41.769027 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0406 19:07:41.769186 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.769429 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:41.771508 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:41.771582 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 19:07:41.771764 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:41.771891 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:41.772985 12299 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0406 19:07:41.773123 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0406 19:07:41.773195 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 19:07:41.773312 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0406 19:07:41.773330 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:41.773457 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:41.773473 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 19:07:41.773701 12299 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0406 19:07:41.773552 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0406 19:07:41.773870 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:41.773909 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 19:07:41.773986 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:41.774117 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:41.774076 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 19:07:41.774239 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0406 19:07:41.774184 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0406 19:07:41.774291 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:41.774341 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:41.774387 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 19:07:41.774376 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.774529 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.774657 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 19:07:41.774699 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.774897 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 19:07:41.774900 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.775046 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:41.775183 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0406 19:07:41.775361 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0406 19:07:41.775536 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0406 19:07:41.775722 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:41.775325 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 19:07:41.775792 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:41.775882 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:41.776103 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:41.776188 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:41.776094 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:41.776233 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 19:07:41.776290 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.776367 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:41.776419 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:41.776509 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:41.776466 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:41.776563 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:41.776611 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:41.776658 12299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0406 19:07:41.776747 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:41.776922 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:41.776997 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:41.777081 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:41.777157 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0406 19:07:41.777308 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:41.777385 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:41.777462 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:41.777609 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:41.777694 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:41.780022 12299 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0406 19:07:41.780250 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } W0406 19:07:41.780437 12299 api.go:53] PageSize not implemented W0406 19:07:41.780506 12299 api.go:53] PageSize not implemented I0406 19:07:41.780439 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:41.780662 12299 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.780693 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 19:07:41.780677 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.780891 12299 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 19:07:41.780934 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.781038 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0406 19:07:41.781152 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:41.781234 12299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 19:07:41.781314 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0406 19:07:41.781305 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:41.781350 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 19:07:41.781371 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0406 19:07:41.781433 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 19:07:41.781555 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 19:07:41.781498 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.781598 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0406 19:07:41.781648 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0406 19:07:41.781667 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:41.781788 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:41.781930 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.782190 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0406 19:07:41.782319 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:41.782406 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 19:07:41.782544 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.782701 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.782839 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.782986 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0406 19:07:41.783120 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.783271 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.783462 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.783613 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I0406 19:07:41.783751 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" }] }] } I0406 19:07:41.783956 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:41.784055 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:41.784134 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 19:07:41.784303 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:41.784482 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.784765 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.784945 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.785087 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.785225 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.785352 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 19:07:41.785484 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.785590 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.785664 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.785824 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.785885 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0406 19:07:41.786014 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:41.786152 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:41.786260 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 19:07:41.786310 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0406 19:07:41.786670 12299 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0406 19:07:41.786880 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:41.786919 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:41.787024 12299 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:41.787065 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:41.787180 12299 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 19:07:41.787359 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 19:07:41.787443 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:41.787525 12299 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0406 19:07:41.787662 12299 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0406 19:07:41.787755 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:41.787898 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0406 19:07:41.787971 12299 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0406 19:07:41.788032 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:41.788089 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:41.788158 12299 dns.go:238] Pre-creating DNS records I0406 19:07:41.788184 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:41.794383 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:41.797429 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.797525 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.797590 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:41.835092 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.180776 12299 iamrole.go:91] ListRoles: { } I0406 19:07:42.183090 12299 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0406 19:07:42.183298 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.183616 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.183735 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:42.183737 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.183903 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0406 19:07:42.184047 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.184204 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:42.184253 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.184450 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:42.184519 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.185873 12299 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0406 19:07:42.186045 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0406 19:07:42.186184 12299 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0406 19:07:42.186294 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0406 19:07:42.186238 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.186348 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0406 19:07:42.186362 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0406 19:07:42.186561 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:42.186627 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.186859 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.186979 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.187119 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0406 19:07:42.187281 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:42.187335 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:42.187387 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.187519 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:42.187570 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.187726 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0406 19:07:42.187869 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:42.187920 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:42.187962 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.188041 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0406 19:07:42.188173 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:42.189934 12299 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0406 19:07:42.190038 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.190133 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0406 19:07:42.190252 12299 api.go:53] PageSize not implemented I0406 19:07:42.190270 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0406 19:07:42.190295 12299 api.go:53] PageSize not implemented I0406 19:07:42.190337 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.190351 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 19:07:42.190470 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:42.190482 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0406 19:07:42.190535 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 19:07:42.190540 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0406 19:07:42.190574 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 19:07:42.190669 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 19:07:42.190712 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0406 19:07:42.190654 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 19:07:42.190849 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.190974 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.191086 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.191224 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.191376 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.191459 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.191592 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.192491 12299 executor.go:111] Tasks: 113 done / 119 total; 5 can run W0406 19:07:42.192599 12299 api.go:53] PageSize not implemented I0406 19:07:42.192622 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 19:07:42.192730 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.192819 12299 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0406 19:07:42.192791 12299 api.go:53] PageSize not implemented I0406 19:07:42.192763 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.192852 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 19:07:42.192883 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:42.192942 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 19:07:42.193009 12299 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0406 19:07:42.193095 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.193206 12299 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0406 19:07:42.196641 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:42.196703 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:42.196742 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:42.196776 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:42.196815 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 19:07:42.196854 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:42.196887 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:42.196920 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:42.196957 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:42.196993 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:42.197031 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:42.197078 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:42.197122 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 19:07:42.197163 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 19:07:42.197200 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:42.197234 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:42.197267 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:42.197304 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:42.197337 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:42.197373 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:42.197408 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:42.197445 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:42.197481 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:42.197517 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:42.197554 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:42.197591 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:42.197624 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:42.197665 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:42.197699 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:42.197733 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:42.197771 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:42.197808 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:42.197843 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:42.199382 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:42.199409 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:42.199448 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199518 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0406 19:07:42.199601 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199675 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199728 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199782 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199823 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199870 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0406 19:07:42.199929 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.199982 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:42.200009 12299 natgateway.go:117] DescribeNatGateways: { } I0406 19:07:42.200047 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 19:07:42.200086 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0406 19:07:42.200135 12299 api.go:53] PageSize not implemented I0406 19:07:42.200152 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 19:07:42.200208 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:42.200237 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:42.200256 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:42.200282 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:42.200300 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:42.200349 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.200377 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:42.200418 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:42.200462 12299 iamrole.go:91] ListRoles: { } I0406 19:07:42.200511 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:42.200558 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:42.200601 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0406 19:07:42.200639 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:42.200671 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0406 19:07:42.201397 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0406 19:07:42.201488 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } autoscaling-group:bastion.privatecalico.example.com ok I0406 19:07:42.201386 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok I0406 19:07:42.201367 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-group:nodes.privatecalico.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0406 19:07:42.201568 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 19:07:42.201606 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0406 19:07:42.201653 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0406 19:07:42.201690 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0406 19:07:42.201507 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0406 19:07:42.201771 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.201741 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 19:07:42.201856 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0406 19:07:42.201805 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I0406 19:07:42.201958 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:42.202040 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 19:07:42.202282 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.202302 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 19:07:42.202363 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-3 ok I0406 19:07:42.202436 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 19:07:42.202504 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 19:07:42.202569 12299 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-1 ok I0406 19:07:42.202641 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:42.202699 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0406 19:07:42.202779 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.202803 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 19:07:42.202866 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 19:07:42.202902 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0406 19:07:42.203015 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0406 19:07:42.203066 12299 iamrole.go:182] ListAttachedRolePolicies: masters.privatecalico.example.com I0406 19:07:42.203032 12299 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0406 19:07:42.203080 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 19:07:42.203155 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0406 19:07:42.203152 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 19:07:42.203203 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0406 19:07:42.203267 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.privatecalico.example.com I0406 19:07:42.203248 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:42.203285 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 19:07:42.203319 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0406 19:07:42.203298 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:42.203364 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0406 19:07:42.203444 12299 iamrole.go:182] ListAttachedRolePolicies: bastions.privatecalico.example.com I0406 19:07:42.203459 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 19:07:42.203393 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0406 19:07:42.203520 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.203539 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:42.203493 12299 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0406 19:07:42.203575 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:42.203624 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 19:07:42.203701 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.203712 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:42.203793 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:42.203861 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 19:07:42.203946 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:42.204038 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:42.204301 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.86s) === RUN TestLifecyclePrivateKopeio I0406 19:07:42.204837 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:42.204883 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:42.204923 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:42.204969 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:42.205013 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:42.205041 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:42.205069 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:42.205101 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:42.205129 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:42.205185 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0406 19:07:42.240835 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:42.244221 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.244321 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.244385 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.283374 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.415620 12299 iamrole.go:91] ListRoles: { } I0406 19:07:42.417904 12299 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0406 19:07:42.418337 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0406 19:07:42.418510 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 19:07:42.418688 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.418748 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0406 19:07:42.418355 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:42.418889 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 19:07:42.418853 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.419125 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.419107 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0406 19:07:42.419161 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.419228 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 19:07:42.419226 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.419415 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.419342 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 19:07:42.419649 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 19:07:42.419790 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:42.419660 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 19:07:42.419844 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:42.419991 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:42.419862 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" }] } I0406 19:07:42.420039 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.420235 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:42.420368 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:42.420432 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:42.420472 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.420667 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:42.420715 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:42.420765 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.421117 12299 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0406 19:07:42.421235 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.421350 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0406 19:07:42.421514 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0406 19:07:42.421439 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0406 19:07:42.421503 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:42.421566 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 19:07:42.421613 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.421797 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0406 19:07:42.421732 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.421866 12299 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0406 19:07:42.421921 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 19:07:42.421942 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 19:07:42.421994 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 19:07:42.422060 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:42.422134 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0406 19:07:42.422268 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 19:07:42.422377 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:42.422483 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:42.422556 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:42.422647 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:42.422615 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 19:07:42.422695 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.422870 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0406 19:07:42.423128 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0406 19:07:42.424021 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] }] } I0406 19:07:42.424453 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0406 19:07:42.424394 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:42.424725 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.424922 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.425083 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.424765 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 19:07:42.425260 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0406 19:07:42.425462 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.425614 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:42.425502 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:42.425721 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:42.425904 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0406 19:07:42.425971 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 19:07:42.426064 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.426233 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:42.426321 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:42.426395 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:42.426550 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:42.426629 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.426480 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:42.426861 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:42.426970 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 19:07:42.427363 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:42.429366 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:42.429380 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0406 19:07:42.429477 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:42.429527 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:42.429625 12299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0406 19:07:42.429719 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:42.429804 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:42.429873 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0406 19:07:42.429923 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0406 19:07:42.429962 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 19:07:42.430001 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:42.430043 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:42.430142 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0406 19:07:42.430205 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:42.430299 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:42.430372 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:42.430417 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:42.430472 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:42.430510 12299 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] } I0406 19:07:42.431810 12299 executor.go:111] Tasks: 79 done / 124 total; 39 can run W0406 19:07:42.432228 12299 api.go:53] PageSize not implemented I0406 19:07:42.432094 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } W0406 19:07:42.432308 12299 api.go:53] PageSize not implemented I0406 19:07:42.432337 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0406 19:07:42.432417 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0406 19:07:42.432481 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.432536 12299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] } I0406 19:07:42.432633 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0406 19:07:42.432687 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 19:07:42.432784 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 19:07:42.432638 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.432929 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.433033 12299 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 19:07:42.433100 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.433024 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.433133 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0406 19:07:42.433176 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0406 19:07:42.433261 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:42.433413 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0406 19:07:42.433579 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.433731 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.433886 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:42.434206 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.434424 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.434540 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.434666 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0406 19:07:42.434808 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 19:07:42.435050 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.435235 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.435419 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:42.435771 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:42.435857 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0406 19:07:42.435924 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 19:07:42.436037 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 19:07:42.436182 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:42.436293 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 19:07:42.436415 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 19:07:42.436542 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" }] }] } I0406 19:07:42.436690 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0406 19:07:42.436734 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 19:07:42.436783 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:42.436823 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:42.436859 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0406 19:07:42.436956 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:42.437084 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.437234 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" }] }] } I0406 19:07:42.437364 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.437491 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.437642 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 19:07:42.437719 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.437851 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 19:07:42.437924 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.438171 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0406 19:07:42.438407 12299 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0406 19:07:42.438490 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 19:07:42.438657 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I0406 19:07:42.438694 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.438825 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 19:07:42.438960 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0406 19:07:42.438993 12299 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0406 19:07:42.439033 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I0406 19:07:42.439064 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.439116 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0406 19:07:42.439147 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.439191 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:42.439240 12299 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0406 19:07:42.439283 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0406 19:07:42.439331 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0406 19:07:42.439430 12299 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0406 19:07:42.439478 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:42.439530 12299 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0406 19:07:42.439627 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0406 19:07:42.441388 12299 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0406 19:07:42.441536 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.441693 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 19:07:42.441708 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.441868 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0406 19:07:42.441931 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.442063 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0406 19:07:42.442128 12299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 19:07:42.442183 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.442215 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:42.442272 12299 dns.go:238] Pre-creating DNS records I0406 19:07:42.442298 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:42.448655 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:42.451670 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.451768 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.451833 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.490208 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.622477 12299 iamrole.go:91] ListRoles: { } I0406 19:07:42.624855 12299 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0406 19:07:42.625097 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.625321 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:42.625383 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:42.625485 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.625686 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.625625 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:42.625792 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.625926 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.626019 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.626187 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I0406 19:07:42.627534 12299 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0406 19:07:42.627680 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0406 19:07:42.627744 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.627906 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.628051 12299 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0406 19:07:42.628027 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.628166 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0406 19:07:42.628184 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.628386 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0406 19:07:42.628452 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0406 19:07:42.628630 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:42.628685 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:42.628733 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.628871 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:42.628917 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0406 19:07:42.629058 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:42.629658 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.629953 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0406 19:07:42.630145 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.630267 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0406 19:07:42.630415 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.630548 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.630717 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.630865 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:42.630946 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0406 19:07:42.631191 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:42.631327 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:42.631593 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:42.636938 12299 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0406 19:07:42.637148 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0406 19:07:42.637388 12299 api.go:53] PageSize not implemented I0406 19:07:42.637402 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0406 19:07:42.637435 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.637581 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.637623 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0406 19:07:42.637573 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0406 19:07:42.637669 12299 api.go:53] PageSize not implemented I0406 19:07:42.637752 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.637706 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.637834 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0406 19:07:42.637883 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 19:07:42.637907 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.638120 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.638419 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0406 19:07:42.638618 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.638928 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 19:07:42.639203 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.639442 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.639641 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:42.639715 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.639834 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.640649 12299 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0406 19:07:42.640736 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.640842 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0406 19:07:42.640883 12299 api.go:53] PageSize not implemented I0406 19:07:42.640858 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.640979 12299 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0406 19:07:42.640939 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } W0406 19:07:42.641090 12299 api.go:53] PageSize not implemented I0406 19:07:42.641132 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.641354 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.641523 12299 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 19:07:42.645941 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:42.646005 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:42.646044 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:42.646080 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:42.646117 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:42.646151 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:42.646184 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:42.646219 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:42.646256 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:42.646292 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:42.646332 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:42.646370 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 19:07:42.646413 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:42.646449 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:42.646485 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:42.646524 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:42.646559 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:42.646596 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0406 19:07:42.646634 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:42.646672 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:42.646709 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:42.646744 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:42.646780 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:42.646815 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:42.646854 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:42.646887 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:42.646925 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 19:07:42.646963 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0406 19:07:42.647001 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:42.647035 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:42.647068 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:42.647105 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:42.647142 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:42.647178 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:42.649159 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:42.649189 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:42.649232 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649302 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0406 19:07:42.649380 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649438 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649486 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649538 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649610 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649661 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0406 19:07:42.649727 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649793 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:42.649827 12299 natgateway.go:117] DescribeNatGateways: { } I0406 19:07:42.649866 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 19:07:42.649911 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0406 19:07:42.649959 12299 api.go:53] PageSize not implemented I0406 19:07:42.649974 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0406 19:07:42.650025 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:42.650052 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:42.650071 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:42.650095 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:42.650113 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:42.650161 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.650188 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:42.650239 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:42.650276 12299 iamrole.go:91] ListRoles: { } I0406 19:07:42.650321 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:42.650369 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:42.650420 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2","rtb-3"] } I0406 19:07:42.650464 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0406 19:07:42.650497 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0406 19:07:42.651163 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:42.651217 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 19:07:42.651295 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } autoscaling-group:nodes.privatekopeio.example.com ok I0406 19:07:42.651323 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0406 19:07:42.651346 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } autoscaling-group:bastion.privatekopeio.example.com ok I0406 19:07:42.651461 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0406 19:07:42.651229 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W0406 19:07:42.651535 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-instance-profile:masters.privatekopeio.example.com ok I0406 19:07:42.651557 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 19:07:42.651632 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 19:07:42.651701 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0406 19:07:42.651665 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:42.651496 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0406 19:07:42.651737 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatekopeio.example.com. ok I0406 19:07:42.651777 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok load-balancer:api-privatekopeio-example-tl2bv8 ok I0406 19:07:42.651881 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:42.651969 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0406 19:07:42.652115 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.652139 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0406 19:07:42.652203 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:42.652255 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0406 19:07:42.652337 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.652358 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0406 19:07:42.652394 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0406 19:07:42.652482 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 19:07:42.652576 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 19:07:42.652718 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0406 19:07:42.652787 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0406 19:07:42.652722 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 19:07:42.652815 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 19:07:42.652871 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0406 19:07:42.652849 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:42.652903 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0406 19:07:42.652966 12299 iamrole.go:182] ListAttachedRolePolicies: masters.privatekopeio.example.com I0406 19:07:42.652923 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 19:07:42.653018 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.653033 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:42.652981 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 19:07:42.653081 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0406 19:07:42.653115 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0406 19:07:42.653144 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:42.653173 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0406 19:07:42.653228 12299 iamrole.go:182] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0406 19:07:42.653241 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 19:07:42.653198 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0406 19:07:42.653296 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:42.653315 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:42.653271 12299 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0406 19:07:42.653370 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0406 19:07:42.653437 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:42.653545 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 19:07:42.653615 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:42.653668 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0406 19:07:42.653754 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:42.653840 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:42.654148 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.45s) === RUN TestLifecycleIPv6 I0406 19:07:42.654673 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:42.654722 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:42.654789 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:42.654840 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:42.654883 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:42.654916 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:42.654947 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:42.654983 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:42.655011 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:42.655068 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0406 19:07:42.679948 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:42.683197 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.683294 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.720633 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.898167 12299 iamrole.go:91] ListRoles: { } I0406 19:07:42.900337 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0406 19:07:42.900810 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.901016 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:42.901069 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.900958 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] } I0406 19:07:42.901089 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.901253 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0406 19:07:42.901325 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0406 19:07:42.901342 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:42.901392 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0406 19:07:42.901546 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:42.901544 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0406 19:07:42.901618 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.901738 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0406 19:07:42.901835 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.901972 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:42.902041 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:42.902089 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:42.902113 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:42.902154 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.902393 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:42.903598 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:42.903713 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.903779 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:42.903963 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.904049 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.904499 12299 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0406 19:07:42.904715 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.904909 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.904958 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 19:07:42.905125 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:42.905138 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 19:07:42.905090 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0406 19:07:42.905269 12299 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0406 19:07:42.905274 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:42.905409 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 19:07:42.905504 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:42.905482 12299 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0406 19:07:42.905829 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0406 19:07:42.905808 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0406 19:07:42.906128 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0406 19:07:42.906382 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] }] } I0406 19:07:42.907060 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0406 19:07:42.907189 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:42.907276 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:42.907319 12299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] } I0406 19:07:42.907314 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 19:07:42.907389 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.908398 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0406 19:07:42.908510 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:42.908548 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0406 19:07:42.908673 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0406 19:07:42.908776 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:42.908870 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:42.908923 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:42.908976 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0406 19:07:42.909060 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:42.909198 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:42.909264 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:42.909313 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:42.909369 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:42.909406 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:42.909450 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:42.909184 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 19:07:42.912266 12299 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0406 19:07:42.912465 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.912630 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.912744 12299 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0406 19:07:42.912805 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0406 19:07:42.913224 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0406 19:07:42.913522 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0406 19:07:42.913659 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:42.913837 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:42.913963 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.914087 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.914195 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:42.914258 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:42.914365 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.914454 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:42.914475 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:42.914706 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.914858 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.914926 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 19:07:42.915101 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.915319 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0406 19:07:42.915701 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.916097 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0406 19:07:42.916489 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.916727 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.916968 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.917217 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 19:07:42.917302 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.917537 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0406 19:07:42.917779 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.918021 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.918287 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 19:07:42.918368 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.918789 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.919260 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.919487 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.919699 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.919869 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:42.919925 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:42.920810 12299 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0406 19:07:42.920964 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:42.921077 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:42.921062 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.921193 12299 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:42.921281 12299 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0406 19:07:42.921385 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.921468 12299 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 19:07:42.921508 12299 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0406 19:07:42.921585 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 19:07:42.921598 12299 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 19:07:42.921644 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:42.921688 12299 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0406 19:07:42.921735 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:42.921792 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0406 19:07:42.921884 12299 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0406 19:07:42.921959 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:42.921990 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.922087 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0406 19:07:42.922149 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.922255 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.922374 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:42.922492 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:42.922557 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:42.922589 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:42.922644 12299 dns.go:238] Pre-creating DNS records I0406 19:07:42.922666 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:42.928341 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:42.931496 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.931590 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:42.969722 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.155515 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.157726 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0406 19:07:43.158194 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.158221 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:43.158371 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.158524 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.158761 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:43.158820 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:43.158922 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.158987 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:43.159095 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:43.159278 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.160650 12299 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0406 19:07:43.160828 12299 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0406 19:07:43.160892 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0406 19:07:43.160851 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0406 19:07:43.161016 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.161175 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0406 19:07:43.161158 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0406 19:07:43.161281 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0406 19:07:43.161374 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:43.161515 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:43.161677 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.162413 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.162684 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0406 19:07:43.163042 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:43.163121 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.163282 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.169752 12299 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0406 19:07:43.170097 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.170242 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.170414 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.170591 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.170772 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.170981 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.171068 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:43.171107 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0406 19:07:43.171145 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 19:07:43.171454 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.171589 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0406 19:07:43.171701 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.171798 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:43.171867 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.174528 12299 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0406 19:07:43.174620 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.174656 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:43.174761 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:43.174782 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 19:07:43.174832 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 19:07:43.174789 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.174887 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:43.174869 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 19:07:43.175047 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0406 19:07:43.175113 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 19:07:43.175341 12299 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0406 19:07:43.175451 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.175502 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:43.175669 12299 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0406 19:07:43.175722 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:43.175768 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 19:07:43.175812 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:43.175937 12299 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0406 19:07:43.175961 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:43.175987 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 19:07:43.176091 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:43.179502 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:43.179563 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:43.179603 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:43.179639 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:43.179677 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 19:07:43.179715 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:43.179749 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:43.179782 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:43.179816 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:43.179853 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:43.179890 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:43.179935 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:43.179974 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 19:07:43.180010 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:43.180043 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:43.180076 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:43.180113 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:43.180146 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:43.180182 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:43.180217 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:43.180252 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:43.180287 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:43.180324 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:43.180358 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:43.180392 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:43.180425 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:43.180463 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:43.180500 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:43.180535 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:43.182134 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:43.182163 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:43.182207 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182265 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0406 19:07:43.182327 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182385 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182435 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182489 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182530 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182574 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0406 19:07:43.182625 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182671 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:43.182698 12299 natgateway.go:117] DescribeNatGateways: { } I0406 19:07:43.182738 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 19:07:43.182780 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0406 19:07:43.182832 12299 api.go:53] PageSize not implemented I0406 19:07:43.182850 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:43.182876 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:43.182890 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 19:07:43.182929 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:43.182953 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:43.182970 12299 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0406 19:07:43.183027 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.183054 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.183101 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:43.183138 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.183181 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:43.183229 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:43.183271 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0406 19:07:43.183309 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:43.183343 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-config:lt-2 ok I0406 19:07:43.183928 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 19:07:43.183992 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0406 19:07:43.183988 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } iam-instance-profile:nodes.minimal-ipv6.example.com ok autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok autoscaling-group:nodes.minimal-ipv6.example.com ok I0406 19:07:43.184129 12299 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0406 19:07:43.183994 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok route53-record:Z1AFAKE1ZON3YO/AAAA/api.minimal-ipv6.example.com ok I0406 19:07:43.184185 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 19:07:43.184252 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0406 19:07:43.184253 12299 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0406 19:07:43.184070 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0406 19:07:43.184400 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:43.184420 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:43.184476 12299 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0406 19:07:43.184542 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:43.184606 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0406 19:07:43.184783 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:43.184803 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 19:07:43.184857 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:43.184925 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 19:07:43.184982 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0406 19:07:43.185062 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:43.185113 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:43.185169 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:43.185203 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 19:07:43.185263 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 19:07:43.185360 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0406 19:07:43.185389 12299 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0406 19:07:43.185492 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:43.185409 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0406 19:07:43.185565 12299 iamrole.go:182] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0406 19:07:43.185581 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 19:07:43.185618 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0406 19:07:43.185652 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0406 19:07:43.185667 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 19:07:43.185695 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0406 19:07:43.185778 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 19:07:43.185870 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:43.185975 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:43.186253 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.53s) === RUN TestLifecycleSharedVPC I0406 19:07:43.186747 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:43.186794 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:43.186832 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:43.186877 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:43.186919 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:43.186949 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:43.186979 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:43.187019 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:43.187047 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:43.187103 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0406 19:07:43.210619 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:43.213395 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.213486 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.257775 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.367483 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.369120 12299 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0406 19:07:43.369496 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:43.369509 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0406 19:07:43.369579 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:43.369659 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0406 19:07:43.369729 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.369935 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0406 19:07:43.369933 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:43.369980 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:43.370045 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0406 19:07:43.370225 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:43.370297 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0406 19:07:43.370319 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 19:07:43.370431 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0406 19:07:43.370557 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.370741 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:43.370884 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:43.370938 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:43.371012 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:43.371042 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:43.372673 12299 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0406 19:07:43.372877 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0406 19:07:43.372942 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 19:07:43.372938 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0406 19:07:43.373109 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 19:07:43.373236 12299 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0406 19:07:43.373212 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] }] } I0406 19:07:43.373429 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0406 19:07:43.373492 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0406 19:07:43.373630 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 19:07:43.373759 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:43.373876 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:43.373939 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:43.373984 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:43.374099 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.374201 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:43.374290 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I0406 19:07:43.374327 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0406 19:07:43.374446 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:43.374534 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:43.374622 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:43.374668 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:43.374718 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] } I0406 19:07:43.374820 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:43.374965 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 19:07:43.377761 12299 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0406 19:07:43.377968 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.378267 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0406 19:07:43.378488 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:43.378632 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.378756 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.378953 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.379239 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.379398 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:43.379603 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 19:07:43.379750 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.379831 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.379919 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.380053 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.380137 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.380359 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.380499 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.380564 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.380852 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 19:07:43.380949 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:43.381031 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 19:07:43.381103 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 19:07:43.381302 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.382456 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.382654 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 19:07:43.382909 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 19:07:43.388392 12299 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0406 19:07:43.388668 12299 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0406 19:07:43.388734 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.388775 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.388841 12299 dns.go:238] Pre-creating DNS records I0406 19:07:43.388870 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.394380 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:43.397514 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.397605 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.436730 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.551738 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.553304 12299 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0406 19:07:43.553799 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:43.553923 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:43.553999 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:43.554170 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:43.554222 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:43.554296 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:43.554500 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.554685 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.556749 12299 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0406 19:07:43.556875 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0406 19:07:43.557057 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:43.557115 12299 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0406 19:07:43.557124 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.557244 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0406 19:07:43.557249 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:43.557346 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0406 19:07:43.557498 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:43.557554 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.557805 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:43.561580 12299 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0406 19:07:43.561728 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.561977 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.562273 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.562398 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.562489 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.562666 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.562831 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.564668 12299 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0406 19:07:43.564880 12299 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0406 19:07:43.567713 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:43.567779 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:43.567816 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:43.567848 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:43.567883 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:43.567918 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:43.567951 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:43.567989 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:43.568028 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:43.568094 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:43.568133 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:43.568167 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:43.568205 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:43.568237 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:43.568273 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:43.568309 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:43.568346 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:43.568379 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:43.568411 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:43.568444 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:43.568481 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:43.568520 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:43.569901 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:43.569932 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:43.569973 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570028 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0406 19:07:43.570086 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570146 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570188 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570232 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570272 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570310 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0406 19:07:43.570359 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570413 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 19:07:43.570449 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0406 19:07:43.570493 12299 api.go:53] PageSize not implemented I0406 19:07:43.570513 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:43.570542 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:43.570560 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:43.570586 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:43.570603 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:43.570656 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.570686 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.570732 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:43.570774 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.570817 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:43.570857 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:43.570887 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.571286 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-config:lt-1 ok autoscaling-group:nodes.sharedvpc.example.com ok volume:vol-1 ok I0406 19:07:43.571370 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0406 19:07:43.571364 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 19:07:43.571449 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } keypair:key-1 ok I0406 19:07:43.571450 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0406 19:07:43.571481 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 19:07:43.571529 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0406 19:07:43.571511 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } iam-instance-profile:masters.sharedvpc.example.com ok iam-instance-profile:nodes.sharedvpc.example.com ok I0406 19:07:43.571565 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 19:07:43.571774 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:43.571790 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:43.571516 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.571825 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0406 19:07:43.571872 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0406 19:07:43.571978 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:43.571991 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:43.572071 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0406 19:07:43.572124 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0406 19:07:43.572138 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 19:07:43.572173 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0406 19:07:43.572204 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0406 19:07:43.572249 12299 iamrole.go:182] ListAttachedRolePolicies: masters.sharedvpc.example.com I0406 19:07:43.572262 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 19:07:43.572287 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0406 19:07:43.572178 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:43.572359 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:43.572676 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.39s) === RUN TestLifecycleComplex I0406 19:07:43.573175 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:43.573220 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:43.573259 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:43.573302 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:43.573343 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:43.573372 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:43.573405 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:43.573438 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:43.573465 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:43.573519 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0406 19:07:43.610673 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:43.614526 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.614637 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.653478 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.764696 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.766601 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0406 19:07:43.766753 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:43.766886 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:43.766996 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:43.767165 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:43.768087 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 19:07:43.768648 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:43.768827 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0406 19:07:43.769065 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } W0406 19:07:43.769504 12299 api.go:53] PageSize not implemented I0406 19:07:43.769418 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.769693 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 19:07:43.770241 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:43.770293 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:43.770339 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:43.770414 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.770469 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.767153 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "nodes.complex.example.com", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0406 19:07:43.770694 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0406 19:07:43.770894 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0406 19:07:43.771293 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0406 19:07:43.771474 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0406 19:07:43.771322 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:43.773093 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0406 19:07:43.773162 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:43.773202 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0406 19:07:43.773262 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:43.773527 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:43.774813 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:43.774964 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:43.775147 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:43.775135 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.775241 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:43.775302 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.775298 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 19:07:43.775349 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0406 19:07:43.775481 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:43.775421 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:43.775440 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.775545 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0406 19:07:43.775712 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.775639 12299 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0406 19:07:43.775698 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0406 19:07:43.775873 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:43.775833 12299 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0406 19:07:43.775940 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.776147 12299 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0406 19:07:43.776205 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.776432 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0406 19:07:43.776481 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:43.776736 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0406 19:07:43.776863 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:43.776978 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0406 19:07:43.777064 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0406 19:07:43.777170 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.777196 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0406 19:07:43.777278 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.777372 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 19:07:43.777471 12299 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0406 19:07:43.777161 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 19:07:43.777517 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:43.777607 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:43.777687 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:43.777734 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0406 19:07:43.777847 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:43.777894 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:43.777931 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:43.777983 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:43.778019 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 19:07:43.778187 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 19:07:43.778176 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0406 19:07:43.778279 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 19:07:43.778390 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:43.778442 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:43.778489 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:43.778621 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:43.778627 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0406 19:07:43.778697 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:43.778747 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:43.778791 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:43.778841 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0406 19:07:43.778949 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:43.778995 12299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0406 19:07:43.779083 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:43.781504 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:43.781962 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:43.782073 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:43.781807 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 19:07:43.782208 12299 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0406 19:07:43.782434 12299 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0406 19:07:43.782577 12299 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0406 19:07:43.782479 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0406 19:07:43.782649 12299 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:43.782682 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 19:07:43.782893 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 19:07:43.783173 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0406 19:07:43.783305 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.783427 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.783531 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.783626 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.783716 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 19:07:43.783784 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0406 19:07:43.783859 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.783976 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.784045 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0406 19:07:43.784155 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 19:07:43.784327 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0406 19:07:43.784542 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I0406 19:07:43.784632 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 19:07:43.784718 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0406 19:07:43.784918 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 19:07:43.785125 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0406 19:07:43.785403 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.785522 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.785587 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 19:07:43.785882 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0406 19:07:43.786090 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0406 19:07:43.786179 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0406 19:07:43.786244 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0406 19:07:43.786378 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 19:07:43.786566 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 19:07:43.786748 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 19:07:43.786923 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0406 19:07:43.787107 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 19:07:43.787293 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:43.787366 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0406 19:07:43.787428 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 19:07:43.787620 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0406 19:07:43.787756 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0406 19:07:43.787886 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0406 19:07:43.788006 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 19:07:43.788214 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0406 19:07:43.788357 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.788507 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.788613 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0406 19:07:43.788841 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 19:07:43.789015 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 19:07:43.789817 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:43.789930 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:43.790100 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:43.790276 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 19:07:43.790344 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.790393 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.790475 12299 dns.go:238] Pre-creating DNS records I0406 19:07:43.790512 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:43.798128 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:43.801955 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.802083 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.840454 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.953561 12299 iamrole.go:91] ListRoles: { } I0406 19:07:43.955880 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0406 19:07:43.956165 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.956349 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0406 19:07:43.956495 12299 api.go:53] PageSize not implemented I0406 19:07:43.956516 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:43.956553 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:43.956617 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.956534 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:43.956688 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.956923 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:43.957072 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:43.957171 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:43.957265 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:43.957359 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.958682 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:43.958825 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.959007 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:43.958997 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:43.959089 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:43.959055 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0406 19:07:43.959151 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:43.959172 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:43.959255 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:43.959257 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:43.959214 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.959423 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.959529 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.959626 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.959763 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.959909 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:43.961108 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:43.961329 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:43.961558 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.961673 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:43.961787 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:43.962006 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:43.962101 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.962357 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:43.962588 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:43.962816 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:43.968756 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:43.969206 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.969277 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:43.969370 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:43.969390 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:43.969446 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:43.969489 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:43.969530 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:43.969569 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:43.969603 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:43.969390 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.969812 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.970017 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.970254 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.970380 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.970490 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:43.970616 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.970684 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:43.970758 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.972380 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:43.972476 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:43.972561 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:43.972584 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:43.972610 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:43.972782 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0406 19:07:43.979102 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:43.982877 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:43.995077 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:43.998736 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:43.998835 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.036364 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.153594 12299 iamrole.go:91] ListRoles: { } I0406 19:07:44.155980 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0406 19:07:44.156523 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0406 19:07:44.156689 12299 api.go:53] PageSize not implemented I0406 19:07:44.156680 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:44.156798 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.156892 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:44.157050 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.157229 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:44.157349 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:44.157452 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.157569 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:44.157601 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:44.157659 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.157709 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.158683 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:44.158825 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:44.158992 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.158935 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:44.159102 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:44.159117 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.159192 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.159194 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:44.159233 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.159277 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:44.159365 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.159387 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:44.159430 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.159563 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.159695 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:44.159746 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:44.159842 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.159952 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.160040 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.160191 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.160330 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:44.160497 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:44.160542 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:44.160662 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.160835 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:44.160912 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:44.164133 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:44.164508 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.164555 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:44.164631 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:44.164605 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.164654 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.164792 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.164844 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.164834 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.164892 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.164945 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.164986 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.165024 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.165197 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0406 19:07:44.165241 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.165205 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.165284 12299 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0406 19:07:44.165418 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.165444 12299 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0406 19:07:44.165509 12299 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0406 19:07:44.165610 12299 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0406 19:07:44.165605 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.165675 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.165712 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.165785 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.166005 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.166227 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.167961 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:44.168095 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:44.168229 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:44.168256 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:44.168281 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.168381 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 19:07:44.168431 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.168460 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:44.168512 12299 dns.go:238] Pre-creating DNS records I0406 19:07:44.180611 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:44.184385 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.184490 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.223730 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.341191 12299 iamrole.go:91] ListRoles: { } I0406 19:07:44.343789 12299 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0406 19:07:44.344007 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0406 19:07:44.344247 12299 api.go:53] PageSize not implemented I0406 19:07:44.344239 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:44.344406 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:44.344530 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:44.344527 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:44.344707 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.344890 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.345037 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.345147 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:44.345178 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:44.345233 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.345279 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:44.346843 12299 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0406 19:07:44.346984 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:44.347114 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:44.347117 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.347252 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:44.347353 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.347237 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.347436 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:44.347499 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.347459 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.347617 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:44.347798 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:44.347878 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:44.348111 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:44.348190 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.348325 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:44.348490 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:44.348554 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.348811 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:44.348882 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0406 19:07:44.349076 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:44.349126 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.349290 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.349388 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.349485 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:44.349531 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.352383 12299 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0406 19:07:44.352576 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.352672 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.352707 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:44.352825 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:44.352855 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.352827 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.352915 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.352956 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.353001 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.353037 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.353070 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.353048 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.353511 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.353677 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.353837 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.353953 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.354146 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.354334 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.356861 12299 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0406 19:07:44.356957 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:44.357044 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:44.357070 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:44.357098 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.357175 12299 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0406 19:07:44.360831 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.364309 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.375857 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:44.379536 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.379634 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.418182 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.530691 12299 iamrole.go:91] ListRoles: { } I0406 19:07:44.533019 12299 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0406 19:07:44.533301 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:44.533435 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:44.533521 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.533669 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:44.533864 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0406 19:07:44.533980 12299 api.go:53] PageSize not implemented I0406 19:07:44.534010 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:44.534239 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 19:07:44.534346 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.534522 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.534655 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.534781 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:44.534816 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:44.534875 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.535986 12299 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0406 19:07:44.536243 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:44.536177 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.536330 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:44.536426 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.536400 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.536518 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:44.536592 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:44.536602 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.536559 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.536693 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.536852 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:44.536896 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.536985 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.537079 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:44.537237 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:44.537291 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:44.537433 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:44.537491 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:44.537717 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:44.537774 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.537862 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.537974 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0406 19:07:44.538094 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.538301 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:44.538400 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:44.545899 12299 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0406 19:07:44.546145 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.546372 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.546446 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:44.546550 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:44.546573 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.546626 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.546582 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.546664 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.546704 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.546739 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.546770 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.546862 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.546984 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.547146 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.547401 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.547529 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.547781 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.548207 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.550705 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.550781 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.553132 12299 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0406 19:07:44.553220 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:44.553308 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:44.553334 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:44.553357 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.553431 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0406 19:07:44.553484 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.553513 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:44.553566 12299 dns.go:238] Pre-creating DNS records I0406 19:07:44.562048 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:44.566343 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.566447 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.604299 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.718848 12299 iamrole.go:91] ListRoles: { } I0406 19:07:44.721159 12299 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0406 19:07:44.721364 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 19:07:44.721631 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:44.721740 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:44.721818 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 19:07:44.722041 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W0406 19:07:44.722079 12299 api.go:53] PageSize not implemented I0406 19:07:44.722103 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:44.722076 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:44.722170 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.722409 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:44.722543 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.722597 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.722751 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.722882 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:44.724452 12299 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0406 19:07:44.724599 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:44.724754 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.724620 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.724902 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:44.724879 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.724885 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:44.724995 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.724961 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:44.724994 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:44.725192 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:44.725259 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.725415 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:44.725458 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.725590 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.725721 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.725848 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:44.725900 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:44.726106 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:44.726153 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.726236 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:44.726349 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 19:07:44.726455 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.726562 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.726689 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:44.726737 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:44.729650 12299 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0406 19:07:44.729823 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:44.729917 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:44.729935 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.729975 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.730012 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.729984 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.730055 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.730092 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.730122 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.730274 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.730499 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.730738 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.730935 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.731111 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.731185 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.731266 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.731333 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.731447 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.733820 12299 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0406 19:07:44.733945 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:44.734078 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:44.734115 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:44.734143 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.734223 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0406 19:07:44.739207 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.742540 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.742631 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.751620 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:44.755363 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.755455 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.755515 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.793486 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.905905 12299 iamrole.go:91] ListRoles: { } I0406 19:07:44.908217 12299 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0406 19:07:44.908676 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0406 19:07:44.908848 12299 api.go:53] PageSize not implemented I0406 19:07:44.908838 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:44.908923 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.909126 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:44.909195 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 19:07:44.909348 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.909508 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:44.909613 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:44.909713 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0406 19:07:44.909854 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.909954 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:44.909984 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:44.910038 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.911573 12299 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0406 19:07:44.911786 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.911866 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:44.911936 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.912005 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:44.912084 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.912069 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:44.912185 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:44.912156 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:44.912253 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:44.912261 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.912328 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.912478 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:44.912522 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.912616 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:44.912658 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.912788 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.912877 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:44.913036 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:44.913149 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:44.913362 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:44.913420 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.913523 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:44.913568 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.913711 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:44.913837 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:44.916994 12299 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0406 19:07:44.917136 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.917212 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:44.917330 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:44.917357 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.917427 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.917432 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.917485 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.917537 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:44.917574 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:44.917606 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:44.917677 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.917955 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.918110 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:44.918286 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:44.918437 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.918611 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.918782 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.918856 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.926158 12299 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0406 19:07:44.926295 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:44.926447 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:44.926493 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:44.926529 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:44.926614 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0406 19:07:44.926674 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:44.926712 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:44.926769 12299 dns.go:238] Pre-creating DNS records I0406 19:07:44.934670 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:44.938897 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.938999 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.939058 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:44.978422 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.094174 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.096490 12299 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0406 19:07:45.096727 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.096942 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0406 19:07:45.097162 12299 api.go:53] PageSize not implemented I0406 19:07:45.097139 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:45.097267 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:45.097225 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.097489 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:45.097550 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:45.097610 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 19:07:45.097769 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:45.097875 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:45.097971 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.098142 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:45.098248 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.099614 12299 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0406 19:07:45.099738 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:45.099829 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:45.099879 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:45.099978 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:45.099977 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:45.099824 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0406 19:07:45.100128 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:45.100104 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:45.100345 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.100396 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.100588 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.100841 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:45.100911 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.101063 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.101210 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.101354 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.101609 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:45.101674 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 19:07:45.101855 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:45.101914 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.102089 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:45.102300 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:45.102501 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.102697 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:45.102776 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:45.104775 12299 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0406 19:07:45.105054 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.105306 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.105343 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:45.105440 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:45.105469 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.105534 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.105531 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.105576 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:45.105632 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:45.105680 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:45.105692 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.105724 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:45.105893 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.106125 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.106294 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.106658 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.106944 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.107047 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.109362 12299 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0406 19:07:45.109478 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:45.109565 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:45.109589 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:45.109614 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.109692 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0406 19:07:45.112659 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.115951 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.116040 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.116098 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.125148 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:45.128855 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.128950 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.129010 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.129064 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.167301 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.282975 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.285310 12299 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0406 19:07:45.285584 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 19:07:45.285817 12299 api.go:53] PageSize not implemented I0406 19:07:45.285959 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.286149 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.286322 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:45.286543 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.286796 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 19:07:45.286972 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.287146 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:45.287200 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:45.287276 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:45.287454 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:45.287567 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 19:07:45.287665 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.288665 12299 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0406 19:07:45.288872 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:45.288970 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:45.289064 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:45.289046 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.289031 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:45.289197 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:45.289291 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:45.289222 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:45.289420 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:45.289446 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:45.289540 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.289712 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.289834 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.289972 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:45.290013 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.290088 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.290194 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.290293 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.290380 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:45.290488 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.290561 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.290666 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.290736 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:45.290775 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:45.290816 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:45.295302 12299 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0406 19:07:45.295485 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.295629 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.295633 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:45.295722 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:45.295740 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.295791 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.295831 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:45.295872 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:45.295836 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.295913 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:45.295947 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:45.296018 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.296147 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.296337 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.296509 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.296952 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.297224 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.297480 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.304660 12299 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0406 19:07:45.304802 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:45.304922 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:45.304958 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:45.304990 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.305140 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0406 19:07:45.305208 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.305248 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:45.305316 12299 dns.go:238] Pre-creating DNS records I0406 19:07:45.313615 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:45.317958 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.318061 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.318144 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.318198 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.355738 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.463319 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.465638 12299 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0406 19:07:45.466728 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:45.465923 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 19:07:45.467443 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.467741 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0406 19:07:45.467961 12299 api.go:53] PageSize not implemented I0406 19:07:45.467925 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 19:07:45.468265 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 19:07:45.468479 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.468781 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.469014 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 19:07:45.469160 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.469291 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:45.469346 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:45.469477 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.473196 12299 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0406 19:07:45.473341 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.473455 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 19:07:45.473541 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:45.473491 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.473555 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:45.473642 12299 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0406 19:07:45.473642 12299 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 19:07:45.473703 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:45.473642 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 19:07:45.473865 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.473945 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.474085 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 19:07:45.474272 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:45.474375 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.474514 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.474667 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.474841 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:45.474895 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.475070 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.475260 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:45.475325 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 19:07:45.475509 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.475645 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.475769 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:45.475831 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 19:07:45.479125 12299 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0406 19:07:45.479309 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.479416 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.479483 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:45.479570 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:45.479588 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.479634 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.479668 12299 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:45.479708 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0406 19:07:45.479744 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0406 19:07:45.479776 12299 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 19:07:45.479709 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.479922 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.480385 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.480749 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.480875 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.481100 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.481310 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:45.481454 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.483739 12299 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0406 19:07:45.483895 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:45.484015 12299 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 19:07:45.484041 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 19:07:45.484069 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.484156 12299 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0406 19:07:45.484521 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:45.484625 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:45.484681 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:45.484763 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:45.484837 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:45.484883 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:45.484919 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:45.484956 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:45.484997 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:45.485047 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:45.485093 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:45.485129 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:45.485164 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:45.485206 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:45.485241 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:45.485281 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:45.485320 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:45.485358 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:45.485397 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:45.485438 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:45.485473 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:45.485512 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 19:07:45.485554 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:45.485589 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:45.485624 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:45.485664 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:45.485713 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:45.485754 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:45.488601 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:45.488634 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:45.488668 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.488729 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0406 19:07:45.488794 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.488852 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.488899 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.488970 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.489014 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.489060 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0406 19:07:45.489119 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.489196 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 19:07:45.489239 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0406 19:07:45.489289 12299 api.go:53] PageSize not implemented I0406 19:07:45.489308 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:45.489335 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:45.489348 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 19:07:45.489389 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:45.489415 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:45.489433 12299 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I0406 19:07:45.489511 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.489536 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:45.489577 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:45.489612 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.489653 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:45.489705 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:45.489752 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } autoscaling-config:lt-1 ok I0406 19:07:45.490380 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0406 19:07:45.490440 12299 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I0406 19:07:45.490454 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 19:07:45.490571 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0406 19:07:45.490511 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:masters.complex.example.com ok volume:vol-2 ok I0406 19:07:45.490582 12299 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok autoscaling-group:nodes.complex.example.com ok I0406 19:07:45.490647 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0406 19:07:45.490700 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.490710 12299 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0406 19:07:45.490538 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok I0406 19:07:45.490750 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0406 19:07:45.490842 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0406 19:07:45.491269 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:45.491295 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0406 19:07:45.491376 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:45.491501 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:45.491556 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 19:07:45.491623 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:45.491676 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:45.491733 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 } ] } W0406 19:07:45.491974 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:45.491995 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 19:07:45.492057 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 19:07:45.492093 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 19:07:45.492177 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0406 19:07:45.492227 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.complex.example.com I0406 19:07:45.492241 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0406 19:07:45.492232 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:45.492279 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0406 19:07:45.492329 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0406 19:07:45.492336 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } I0406 19:07:45.492371 12299 iamrole.go:182] ListAttachedRolePolicies: masters.complex.example.com subnet:subnet-1 ok I0406 19:07:45.492384 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 19:07:45.492412 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0406 19:07:45.492479 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 19:07:45.492565 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:45.492653 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:45.492937 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.92s) === RUN TestLifecycleExternalLB I0406 19:07:45.493445 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:45.493491 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:45.493529 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:45.493572 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:45.493612 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:45.493640 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:45.493671 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:45.493702 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:45.493730 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:45.493799 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0406 19:07:45.521997 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:45.525305 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.525436 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.563506 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.678433 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.680136 12299 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0406 19:07:45.680333 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:45.680425 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:45.680499 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:45.680545 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0406 19:07:45.680657 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] } W0406 19:07:45.680743 12299 api.go:53] PageSize not implemented I0406 19:07:45.680695 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 19:07:45.680812 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0406 19:07:45.680905 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 19:07:45.680916 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.681011 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0406 19:07:45.681057 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.681069 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0406 19:07:45.681151 12299 api.go:53] PageSize not implemented I0406 19:07:45.681213 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.681112 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } W0406 19:07:45.681294 12299 api.go:53] PageSize not implemented I0406 19:07:45.681304 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.681213 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.681369 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.681296 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0406 19:07:45.681426 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:45.681656 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:45.681721 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.681910 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.681916 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.682069 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:45.682146 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:45.682245 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:45.682414 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.682618 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:45.682755 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:45.682815 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:45.682850 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:45.682877 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:45.682916 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.683048 12299 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0406 19:07:45.683191 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.683287 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0406 19:07:45.683425 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:45.683404 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 19:07:45.683528 12299 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0406 19:07:45.683485 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.683619 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:45.683640 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 19:07:45.683690 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0406 19:07:45.683777 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 19:07:45.683896 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:45.683978 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0406 19:07:45.684071 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.684088 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 19:07:45.684274 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:45.684352 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.684579 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.684669 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.684804 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0406 19:07:45.684914 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:45.685013 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:45.685074 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.684974 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 19:07:45.685121 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:45.685206 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:45.685248 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:45.685286 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:45.685343 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:45.685390 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:45.685281 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 19:07:45.688112 12299 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0406 19:07:45.688306 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.688490 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.688608 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.688678 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.688863 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:45.688927 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.689188 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.689481 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" }] }] } I0406 19:07:45.689648 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.689734 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.689804 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.689986 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.690156 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.690290 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.690424 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.690566 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:45.690618 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 19:07:45.690658 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.690797 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:45.690862 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.691001 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 19:07:45.691135 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.691239 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.691330 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:45.691532 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 19:07:45.692541 12299 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0406 19:07:45.692820 12299 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0406 19:07:45.692885 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.692923 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:45.692988 12299 dns.go:238] Pre-creating DNS records I0406 19:07:45.693017 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:45.703927 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:45.706783 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.706889 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.745502 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.851967 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.853284 12299 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0406 19:07:45.854942 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.855078 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:45.855123 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:45.855170 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.855352 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 19:07:45.855444 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 19:07:45.855901 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:45.856002 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:45.855740 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } I0406 19:07:45.856495 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.856657 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.856511 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.856608 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 19:07:45.857000 12299 api.go:53] PageSize not implemented W0406 19:07:45.857126 12299 api.go:53] PageSize not implemented I0406 19:07:45.857262 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 19:07:45.857453 12299 api.go:53] PageSize not implemented I0406 19:07:45.857440 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:45.857513 12299 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.859846 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.857905 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.861963 12299 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0406 19:07:45.862095 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.862282 12299 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0406 19:07:45.862208 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0406 19:07:45.862378 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0406 19:07:45.862435 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.862698 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:45.862770 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0406 19:07:45.863028 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:45.863089 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:45.863228 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.863358 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.863435 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.869364 12299 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0406 19:07:45.869760 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.869936 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.870486 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.870697 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.870770 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.870924 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.871138 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.873028 12299 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0406 19:07:45.873299 12299 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0406 19:07:45.876629 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:45.876693 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:45.876730 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:45.876762 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:45.876803 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:45.876839 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:45.876873 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:45.876909 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:45.876943 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:45.876981 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:45.877036 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:45.877074 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:45.877107 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:45.877141 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:45.877177 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:45.877210 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:45.877245 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:45.877279 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:45.877319 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:45.877353 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:45.877386 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:45.877418 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:45.877455 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:45.877492 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:45.877530 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:45.878952 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:45.878983 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:45.879022 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879074 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0406 19:07:45.879130 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879188 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879236 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879292 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879342 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879384 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0406 19:07:45.879431 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879484 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 19:07:45.879523 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0406 19:07:45.879572 12299 api.go:53] PageSize not implemented I0406 19:07:45.879592 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:45.879619 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:45.879636 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:45.879660 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:45.879675 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:45.879723 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:45.879751 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:45.879798 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:45.879835 12299 iamrole.go:91] ListRoles: { } I0406 19:07:45.879875 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:45.879918 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:45.879952 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:45.880427 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 19:07:45.880493 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0406 19:07:45.880489 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.externallb.example.com ok iam-instance-profile:masters.externallb.example.com ok I0406 19:07:45.880578 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0406 19:07:45.880686 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0406 19:07:45.880900 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:45.880882 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 19:07:45.880937 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:45.880958 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0406 19:07:45.880671 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I0406 19:07:45.880995 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 19:07:45.881102 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 19:07:45.881167 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0406 19:07:45.881275 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:45.881353 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0406 19:07:45.881653 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:45.881679 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0406 19:07:45.881769 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:45.881915 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0406 19:07:45.881921 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:45.882002 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.externallb.example.com I0406 19:07:45.882024 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 19:07:45.882067 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0406 19:07:45.882105 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0406 19:07:45.882177 12299 iamrole.go:182] ListAttachedRolePolicies: masters.externallb.example.com I0406 19:07:45.882190 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 19:07:45.882217 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0406 19:07:45.882287 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:45.882366 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:45.882441 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:45.882710 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.39s) === RUN TestLifecycleSharedSubnet I0406 19:07:45.883198 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:45.883246 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:45.883288 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:45.883333 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:45.883376 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:45.883404 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:45.883432 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:45.883466 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:45.883493 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:45.883569 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0406 19:07:45.911463 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:45.914495 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.914684 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:45.954418 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.071690 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.073173 12299 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0406 19:07:46.073605 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:46.073575 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0406 19:07:46.073777 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0406 19:07:46.073752 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:46.073862 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] } I0406 19:07:46.073908 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:46.073972 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:46.074004 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0406 19:07:46.074029 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.074131 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0406 19:07:46.074275 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] } I0406 19:07:46.074251 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:46.074415 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:46.074472 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.074634 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:46.074790 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:46.074842 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.075909 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.076551 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 19:07:46.078534 12299 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0406 19:07:46.078935 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 19:07:46.079231 12299 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0406 19:07:46.079422 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0406 19:07:46.079178 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:46.079881 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0406 19:07:46.080889 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.080997 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:46.079973 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 19:07:46.081944 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0406 19:07:46.082105 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:46.082112 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 19:07:46.082200 12299 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" } ] } I0406 19:07:46.082335 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:46.082310 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 19:07:46.082418 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.082557 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:46.082635 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.084099 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 19:07:46.086807 12299 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0406 19:07:46.087088 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I0406 19:07:46.087435 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:46.087753 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.087879 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.088129 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.088409 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.088527 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 19:07:46.088993 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" }] }] } I0406 19:07:46.089263 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 19:07:46.089412 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:46.089550 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0406 19:07:46.089685 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 19:07:46.089815 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0406 19:07:46.090375 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 19:07:46.090520 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 19:07:46.090670 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 19:07:46.090829 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 19:07:46.090961 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 19:07:46.091802 12299 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0406 19:07:46.092093 12299 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0406 19:07:46.092170 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.092209 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.092272 12299 dns.go:238] Pre-creating DNS records I0406 19:07:46.092302 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.097930 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:46.100821 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.100916 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.138670 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.251183 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.252782 12299 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0406 19:07:46.252970 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.253235 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:46.253412 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:46.253454 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:46.254958 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:46.253565 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.255719 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.255854 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.260247 12299 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0406 19:07:46.260362 12299 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0406 19:07:46.260377 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 19:07:46.260569 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:46.260641 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0406 19:07:46.260671 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0406 19:07:46.260983 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:46.261073 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0406 19:07:46.261288 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.265824 12299 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0406 19:07:46.265985 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.266508 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.266959 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.267185 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:46.269008 12299 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0406 19:07:46.269353 12299 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0406 19:07:46.272171 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:46.272237 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:46.272273 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:46.272305 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:46.272340 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:46.272376 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:46.272410 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:46.272448 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:46.272488 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:46.272530 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:46.272563 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:46.272593 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:46.272625 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:46.272661 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:46.272698 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:46.272737 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:46.272770 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:46.272804 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:46.272842 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:46.272879 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:46.274306 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:46.274336 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:46.274375 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274428 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0406 19:07:46.274486 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274546 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274590 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274632 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274675 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274712 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0406 19:07:46.274776 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274822 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 19:07:46.274857 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0406 19:07:46.274901 12299 api.go:53] PageSize not implemented I0406 19:07:46.274924 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:46.274954 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:46.274974 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:46.275001 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:46.275021 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:46.275071 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.275098 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.275146 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:46.275182 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.275233 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:46.275272 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:46.275661 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.sharedsubnet.example.com ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0406 19:07:46.275810 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 19:07:46.275890 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0406 19:07:46.275927 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 19:07:46.275973 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0406 19:07:46.275724 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 19:07:46.276034 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.276061 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:46.275846 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok I0406 19:07:46.276113 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 19:07:46.276170 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0406 19:07:46.276295 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.276380 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 19:07:46.276541 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.276562 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:46.276597 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:46.276700 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0406 19:07:46.276750 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0406 19:07:46.276764 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 19:07:46.276800 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0406 19:07:46.276831 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0406 19:07:46.276873 12299 iamrole.go:182] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0406 19:07:46.276885 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 19:07:46.276916 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0406 19:07:46.277171 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedSubnet I0406 19:07:46.277695 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:46.277740 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:46.277779 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:46.277825 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:46.277865 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:46.277895 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:46.277926 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:46.277958 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:46.277983 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:46.278037 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0406 19:07:46.306110 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:46.309064 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.309152 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.309218 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.347143 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.460283 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.462426 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0406 19:07:46.462683 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0406 19:07:46.462773 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 19:07:46.462860 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:46.462968 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0406 19:07:46.463018 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:46.463071 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:46.463070 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 19:07:46.463144 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.463152 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:46.463216 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.463280 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 19:07:46.463335 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.463586 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 19:07:46.463594 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.463845 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0406 19:07:46.463873 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.464030 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] } I0406 19:07:46.464122 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:46.464283 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:46.464199 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" }] } I0406 19:07:46.464337 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:46.464454 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:46.467431 12299 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0406 19:07:46.468335 12299 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0406 19:07:46.468526 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 19:07:46.468407 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0406 19:07:46.468740 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0406 19:07:46.468908 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0406 19:07:46.468810 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0406 19:07:46.469546 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 19:07:46.469802 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 19:07:46.469802 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0406 19:07:46.470454 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 19:07:46.471135 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.471185 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 19:07:46.472491 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 19:07:46.472433 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0406 19:07:46.472725 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0406 19:07:46.472963 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0406 19:07:46.473217 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:46.473381 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.473500 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:46.473572 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:46.473634 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:46.473692 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.473804 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:46.473861 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:46.473903 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.473995 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:46.474047 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:46.474095 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.474253 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:46.474324 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:46.474425 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.474472 12299 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Utility" },{ Key: "kubernetes.io/role/elb", Value: "1" }] } I0406 19:07:46.474547 12299 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" },{ Key: "kubernetes.io/role/internal-elb", Value: "1" }] } I0406 19:07:46.474156 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 19:07:46.474623 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:46.474673 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:46.477366 12299 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0406 19:07:46.477583 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0406 19:07:46.477788 12299 api.go:53] PageSize not implemented W0406 19:07:46.477844 12299 api.go:53] PageSize not implemented I0406 19:07:46.477734 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 19:07:46.478006 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0406 19:07:46.477995 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.478171 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.478193 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0406 19:07:46.478278 12299 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 19:07:46.478371 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.478409 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0406 19:07:46.478468 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0406 19:07:46.478407 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I0406 19:07:46.478580 12299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0406 19:07:46.478679 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0406 19:07:46.478735 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0406 19:07:46.478813 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0406 19:07:46.478788 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.479084 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" }] }] } I0406 19:07:46.479336 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:46.479517 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.479664 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.479811 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.479939 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:46.480049 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 19:07:46.480171 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:46.480286 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.480379 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.480532 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0406 19:07:46.480663 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.480789 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.480940 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.481070 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" }] }] } I0406 19:07:46.481210 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.481446 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.481682 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 19:07:46.481826 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.481906 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.481986 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 19:07:46.482129 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.482171 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.482285 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.484574 12299 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0406 19:07:46.484675 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:46.484850 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:46.485095 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:46.485175 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.485211 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.485274 12299 dns.go:238] Pre-creating DNS records I0406 19:07:46.485298 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.491625 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:46.494587 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.494679 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.494764 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.535237 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.649466 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.651717 12299 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0406 19:07:46.651956 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.652154 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.652280 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.652476 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:46.652558 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:46.652596 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:46.652631 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:46.652684 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.654476 12299 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0406 19:07:46.654656 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0406 19:07:46.654787 12299 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0406 19:07:46.654707 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:46.654907 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0406 19:07:46.655026 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0406 19:07:46.655135 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:46.655241 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0406 19:07:46.655426 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.655484 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 19:07:46.655632 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0406 19:07:46.655812 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:46.655871 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0406 19:07:46.656012 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:46.656066 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0406 19:07:46.656269 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:46.656363 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0406 19:07:46.664682 12299 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0406 19:07:46.664904 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0406 19:07:46.665043 12299 api.go:53] PageSize not implemented I0406 19:07:46.665061 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.665073 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.665195 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.665244 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } W0406 19:07:46.665295 12299 api.go:53] PageSize not implemented I0406 19:07:46.665284 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.665345 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.665498 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.665511 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0406 19:07:46.665678 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0406 19:07:46.665764 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.665980 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.667809 12299 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0406 19:07:46.667910 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:46.667997 12299 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0406 19:07:46.668026 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0406 19:07:46.668089 12299 api.go:53] PageSize not implemented I0406 19:07:46.668136 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } W0406 19:07:46.668356 12299 api.go:53] PageSize not implemented I0406 19:07:46.668393 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.668593 12299 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0406 19:07:46.672156 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:46.672217 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:46.672253 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:46.672285 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:46.672320 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:46.672352 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:46.672385 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:46.672423 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:46.672464 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:46.672502 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 19:07:46.672543 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:46.672576 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:46.672606 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:46.672637 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:46.672672 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:46.672708 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:46.672742 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:46.672776 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:46.672811 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:46.672847 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:46.672883 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:46.672916 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:46.672952 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:46.672988 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:46.674542 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:46.674569 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:46.674606 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.674671 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0406 19:07:46.674770 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.674835 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.674877 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.674918 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.674959 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.674994 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0406 19:07:46.675030 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.675071 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 19:07:46.675105 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0406 19:07:46.675144 12299 api.go:53] PageSize not implemented I0406 19:07:46.675158 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0406 19:07:46.675209 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:46.675236 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:46.675254 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:46.675278 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:46.675294 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 19:07:46.675342 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.675369 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.675414 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:46.675454 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.675497 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:46.675538 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:46.676056 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0406 19:07:46.676137 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:nodes.private-shared-subnet.example.com ok I0406 19:07:46.676143 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0406 19:07:46.676282 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 19:07:46.676392 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0406 19:07:46.676237 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.676392 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0406 19:07:46.676231 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } load-balancer:bastion-private-shared-su-5ol32q ok W0406 19:07:46.676521 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.676472 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 19:07:46.676547 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 19:07:46.676574 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0406 19:07:46.676627 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0406 19:07:46.676631 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } volume:vol-2 ok I0406 19:07:46.676690 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0406 19:07:46.676699 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0406 19:07:46.676775 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:46.676872 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0406 19:07:46.677002 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.677043 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:46.677190 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0406 19:07:46.677472 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.677528 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0406 19:07:46.677619 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 19:07:46.677832 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0406 19:07:46.677906 12299 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0406 19:07:46.677819 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.677923 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 19:07:46.677963 12299 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0406 19:07:46.678000 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0406 19:07:46.678047 12299 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0406 19:07:46.677961 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 19:07:46.678113 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.678129 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:46.678059 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 19:07:46.678170 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:46.678189 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0406 19:07:46.678250 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0406 19:07:46.678226 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0406 19:07:46.678332 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:46.678350 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0406 19:07:46.678312 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0406 19:07:46.678394 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 19:07:46.678429 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0406 19:07:46.678727 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.40s) === RUN TestLifecyclePrivateSharedIP I0406 19:07:46.679234 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:46.679285 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:46.679329 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:46.679378 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:46.679422 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:46.679456 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:46.679487 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:46.679522 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:46.679546 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:46.679602 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0406 19:07:46.708303 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:46.711226 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.711318 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.711380 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.749847 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.865351 12299 iamrole.go:91] ListRoles: { } I0406 19:07:46.867556 12299 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0406 19:07:46.868025 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.868337 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0406 19:07:46.868393 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.868423 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.868387 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:46.868468 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:46.868554 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0406 19:07:46.868620 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" }] } I0406 19:07:46.868626 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.868697 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0406 19:07:46.868787 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:46.868854 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.868960 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:46.868960 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.869363 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.869198 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:46.870174 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:46.870404 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:46.870501 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:46.871244 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:46.871384 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:46.871476 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:46.875204 12299 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0406 19:07:46.875368 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0406 19:07:46.875453 12299 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0406 19:07:46.875392 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:46.875550 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0406 19:07:46.875566 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0406 19:07:46.875637 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 19:07:46.875695 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0406 19:07:46.875854 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 19:07:46.875864 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:46.876022 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:46.875995 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 19:07:46.876168 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0406 19:07:46.876190 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:46.876280 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 19:07:46.876364 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 19:07:46.876430 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0406 19:07:46.876509 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0406 19:07:46.876744 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:46.876866 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:46.876970 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 19:07:46.877088 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0406 19:07:46.877113 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0406 19:07:46.877350 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0406 19:07:46.877573 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.877676 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0406 19:07:46.877793 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.877904 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.878022 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:46.878089 12299 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0406 19:07:46.878167 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:46.878221 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:46.878260 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:46.878306 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:46.878344 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.878431 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:46.878475 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:46.878515 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:46.878621 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:46.878674 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:46.878270 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 19:07:46.878729 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:46.878773 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:46.878823 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0406 19:07:46.878910 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:46.878967 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:46.879005 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 19:07:46.879100 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:46.879145 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:46.880291 12299 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0406 19:07:46.880473 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } W0406 19:07:46.880617 12299 api.go:53] PageSize not implemented I0406 19:07:46.880661 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0406 19:07:46.880746 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } W0406 19:07:46.880768 12299 api.go:53] PageSize not implemented I0406 19:07:46.880938 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 19:07:46.880884 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.881030 12299 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.881130 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0406 19:07:46.881086 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.881229 12299 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 19:07:46.881241 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.881300 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0406 19:07:46.881343 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.881335 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 19:07:46.881483 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 19:07:46.881572 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 19:07:46.881515 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0406 19:07:46.881640 12299 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0406 19:07:46.881679 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:46.881695 12299 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0406 19:07:46.881767 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:46.881869 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.882266 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.882538 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:46.882697 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:46.882796 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.882954 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:46.882995 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.883133 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 19:07:46.883226 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 19:07:46.883345 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 19:07:46.883483 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:46.883524 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 19:07:46.883568 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.883694 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.883852 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.883966 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" }] }] } I0406 19:07:46.884195 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 19:07:46.884266 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 19:07:46.884412 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0406 19:07:46.884514 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.884749 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.885013 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.885262 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.885539 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.885726 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.885855 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 19:07:46.885980 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0406 19:07:46.886110 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0406 19:07:46.886251 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.886369 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.886435 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 19:07:46.886511 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0406 19:07:46.886557 12299 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0406 19:07:46.886660 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0406 19:07:46.887522 12299 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0406 19:07:46.887745 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:46.887820 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:46.887938 12299 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:46.888135 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:46.888290 12299 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 19:07:46.888464 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 19:07:46.888523 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:46.888566 12299 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0406 19:07:46.888635 12299 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0406 19:07:46.888697 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:46.888820 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0406 19:07:46.888889 12299 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0406 19:07:46.888940 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:46.888971 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.889024 12299 dns.go:238] Pre-creating DNS records I0406 19:07:46.889048 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:46.895185 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0406 19:07:46.898125 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.898216 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.898284 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:46.936162 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:47.052704 12299 iamrole.go:91] ListRoles: { } I0406 19:07:47.054944 12299 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0406 19:07:47.055377 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:47.055500 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:47.055646 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 19:07:47.055736 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 19:07:47.055772 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 19:07:47.055843 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:47.055852 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.056057 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.058605 12299 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0406 19:07:47.058895 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0406 19:07:47.058768 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0406 19:07:47.058979 12299 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0406 19:07:47.059030 12299 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0406 19:07:47.059008 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.059181 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0406 19:07:47.059480 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:47.059577 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.059803 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:47.059864 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0406 19:07:47.060657 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:47.060888 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0406 19:07:47.061330 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 19:07:47.061630 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.061883 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 19:07:47.061971 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:47.062023 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0406 19:07:47.062228 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.062405 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 19:07:47.062516 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:47.066855 12299 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0406 19:07:47.067057 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0406 19:07:47.067191 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } W0406 19:07:47.067264 12299 api.go:53] PageSize not implemented W0406 19:07:47.067298 12299 api.go:53] PageSize not implemented I0406 19:07:47.067340 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 19:07:47.067381 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:47.067457 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 19:07:47.067507 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0406 19:07:47.067553 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.067580 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 19:07:47.067697 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0406 19:07:47.067738 12299 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 19:07:47.067768 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.067914 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:47.068156 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:47.068457 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:47.068640 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.068859 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:47.069050 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:47.069185 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 19:07:47.071194 12299 executor.go:111] Tasks: 108 done / 114 total; 5 can run W0406 19:07:47.071422 12299 api.go:53] PageSize not implemented I0406 19:07:47.071363 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:47.071526 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:47.071475 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 19:07:47.071478 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 19:07:47.071680 12299 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0406 19:07:47.071751 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0406 19:07:47.071899 12299 api.go:53] PageSize not implemented I0406 19:07:47.071942 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 19:07:47.072250 12299 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0406 19:07:47.072328 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 19:07:47.072435 12299 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0406 19:07:47.076322 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:47.076383 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:47.076419 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:47.076459 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:47.076525 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:47.076563 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:47.076596 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:47.076635 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:47.076676 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:47.076721 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 19:07:47.076760 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 19:07:47.076795 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:47.076826 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:47.076862 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:47.076898 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:47.076931 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 19:07:47.076968 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:47.077003 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:47.077037 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 19:07:47.077073 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 19:07:47.077109 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 19:07:47.077144 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:47.077180 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:47.077212 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 19:07:47.077249 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:47.077282 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:47.077314 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:47.077351 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:47.077388 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:47.078980 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:47.079007 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:47.079044 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079109 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0406 19:07:47.079193 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079252 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079291 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079333 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079372 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079415 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0406 19:07:47.079469 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079522 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:47.079551 12299 natgateway.go:117] DescribeNatGateways: { } I0406 19:07:47.079591 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 19:07:47.079628 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0406 19:07:47.079670 12299 api.go:53] PageSize not implemented I0406 19:07:47.079684 12299 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 19:07:47.079735 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:47.079763 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:47.079781 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:47.079805 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:47.079822 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 19:07:47.079869 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:47.079894 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:47.079936 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:47.079973 12299 iamrole.go:91] ListRoles: { } I0406 19:07:47.080020 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:47.080067 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:47.080099 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0406 19:07:47.080134 12299 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 19:07:47.080170 12299 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0406 19:07:47.080784 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok autoscaling-group:bastion.private-shared-ip.example.com ok I0406 19:07:47.080849 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0406 19:07:47.080941 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:47.080905 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 19:07:47.080926 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0406 19:07:47.080983 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } load-balancer:api-private-shared-ip-exa-ohatqj ok I0406 19:07:47.080962 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:nodes.private-shared-ip.example.com ok I0406 19:07:47.081053 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:47.081033 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 19:07:47.081124 12299 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 19:07:47.081143 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } load-balancer:bastion-private-shared-ip-eepmph ok iam-instance-profile:masters.private-shared-ip.example.com ok iam-instance-profile:nodes.private-shared-ip.example.com ok I0406 19:07:47.081114 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0406 19:07:47.081188 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 19:07:47.081260 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok W0406 19:07:47.081243 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:47.081334 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:47.081179 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0406 19:07:47.081386 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:47.081480 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 19:07:47.081541 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 19:07:47.081770 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:47.081789 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 19:07:47.081833 12299 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-2 ok I0406 19:07:47.081899 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-3 ok I0406 19:07:47.081950 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0406 19:07:47.082073 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:47.082122 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0406 19:07:47.082132 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 19:07:47.082211 12299 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0406 19:07:47.082230 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 19:07:47.082275 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0406 19:07:47.082310 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0406 19:07:47.082234 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 19:07:47.082376 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-ip.example.com W0406 19:07:47.082392 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:47.082426 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0406 19:07:47.082392 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 19:07:47.082472 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0406 19:07:47.082467 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 19:07:47.082512 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0406 19:07:47.082558 12299 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0406 19:07:47.082571 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 19:07:47.082600 12299 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0406 19:07:47.082543 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 19:07:47.082648 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:47.082666 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0406 19:07:47.082751 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 19:07:47.082893 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:47.082991 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 19:07:47.083289 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0406 19:07:47.083845 12299 internetgateways.go:66] CreateInternetGateway: { } I0406 19:07:47.083905 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:07:47.083955 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:07:47.084029 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:07:47.084086 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:07:47.084136 12299 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:07:47.084178 12299 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:07:47.084218 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:07:47.084258 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:07:47.084324 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0406 19:07:47.109048 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0406 19:07:47.111993 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.112079 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.153332 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:47.283877 12299 iamrole.go:91] ListRoles: { } I0406 19:07:47.285441 12299 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0406 19:07:47.285659 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.285775 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:47.285814 12299 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0406 19:07:47.285795 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0406 19:07:47.285899 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:47.285968 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.286096 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0406 19:07:47.286122 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:47.286183 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0406 19:07:47.286348 12299 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0406 19:07:47.286428 12299 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0406 19:07:47.286249 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:47.287582 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.287693 12299 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0406 19:07:47.287808 12299 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.287883 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.288466 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 19:07:47.288576 12299 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 19:07:47.288646 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:47.288729 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:47.288780 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:47.288853 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:47.288952 12299 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 19:07:47.289196 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:47.289292 12299 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.290694 12299 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0406 19:07:47.290831 12299 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0406 19:07:47.291048 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.291054 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 19:07:47.291946 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0406 19:07:47.291905 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0406 19:07:47.292206 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:47.292187 12299 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 19:07:47.292450 12299 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 19:07:47.292347 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 19:07:47.292512 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:47.292706 12299 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:07:47.292791 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0406 19:07:47.293039 12299 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.293213 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:47.293308 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.293470 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 19:07:47.293564 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.293679 12299 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0406 19:07:47.293716 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I0406 19:07:47.293879 12299 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 19:07:47.294050 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:47.293877 12299 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 19:07:47.294153 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:47.294212 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:47.294251 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:47.294336 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:47.294429 12299 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] } I0406 19:07:47.294564 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:47.294643 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:47.294282 12299 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 19:07:47.296705 12299 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0406 19:07:47.296920 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.297112 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.297325 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 19:07:47.297379 12299 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 19:07:47.297500 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.297742 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.297963 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I0406 19:07:47.298141 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.298349 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.298502 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.298568 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.298714 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.298926 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.299096 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.299244 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.299433 12299 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:47.299488 12299 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 19:07:47.299536 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" }] }] } I0406 19:07:47.299701 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.299843 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.299952 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.300035 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" }] }] } I0406 19:07:47.300264 12299 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.300450 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.300632 12299 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 19:07:47.301501 12299 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0406 19:07:47.301781 12299 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0406 19:07:47.301979 12299 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0406 19:07:47.302059 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:47.302095 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:47.302154 12299 dns.go:238] Pre-creating DNS records I0406 19:07:47.302182 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:47.308251 12299 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0406 19:07:47.311113 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.311203 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.349562 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:47.487053 12299 iamrole.go:91] ListRoles: { } I0406 19:07:47.488863 12299 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0406 19:07:47.489301 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.489522 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:47.489634 12299 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 19:07:47.489600 12299 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 19:07:47.489756 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.489943 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.490078 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.490186 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:07:47.490220 12299 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:07:47.490268 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:47.492440 12299 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0406 19:07:47.492671 12299 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0406 19:07:47.492681 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.492868 12299 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0406 19:07:47.492923 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0406 19:07:47.493147 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 19:07:47.493239 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:47.493344 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.493487 12299 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:07:47.493580 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.493732 12299 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:07:47.493789 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0406 19:07:47.494050 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:47.497418 12299 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0406 19:07:47.497564 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.497742 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.497837 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.497994 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.498155 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.498301 12299 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 19:07:47.498413 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.499803 12299 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0406 19:07:47.500027 12299 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0406 19:07:47.500237 12299 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0406 19:07:47.503101 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 19:07:47.503165 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 19:07:47.503202 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 19:07:47.503236 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 19:07:47.503273 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 19:07:47.503306 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 19:07:47.503339 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 19:07:47.503373 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 19:07:47.503408 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 19:07:47.503442 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 19:07:47.503480 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 19:07:47.503524 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 19:07:47.503557 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 19:07:47.503589 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 19:07:47.503624 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 19:07:47.503656 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:07:47.503691 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 19:07:47.503724 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:07:47.503763 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 19:07:47.503795 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 19:07:47.503827 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 19:07:47.503859 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 19:07:47.503896 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 19:07:47.503932 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:07:47.503966 12299 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 19:07:47.505428 12299 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 19:07:47.505455 12299 keypairs.go:95] DescribeKeyPairs: { } I0406 19:07:47.505490 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505545 12299 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505602 12299 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505659 12299 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505706 12299 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505765 12299 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505807 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505847 12299 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505896 12299 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505950 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 19:07:47.505989 12299 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0406 19:07:47.506039 12299 api.go:53] PageSize not implemented I0406 19:07:47.506058 12299 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 19:07:47.506086 12299 loadbalancers.go:34] PageSize not implemented I0406 19:07:47.506103 12299 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 19:07:47.506127 12299 targetgroups.go:35] PageSize not implemented I0406 19:07:47.506146 12299 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 19:07:47.506191 12299 zones.go:93] ListHostedZonesPages { } I0406 19:07:47.506218 12299 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 19:07:47.506265 12299 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0406 19:07:47.506298 12299 iamrole.go:91] ListRoles: { } I0406 19:07:47.506343 12299 internetgateways.go:101] DescribeInternetGateways: { } I0406 19:07:47.506383 12299 routetable.go:53] DescribeRouteTables: { } I0406 19:07:47.506414 12299 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 19:07:47.506872 12299 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0406 19:07:47.506937 12299 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 19:07:47.506966 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0406 19:07:47.507044 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 19:07:47.507157 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I0406 19:07:47.507015 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } I0406 19:07:47.506997 12299 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } W0406 19:07:47.507215 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I0406 19:07:47.507235 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 19:07:47.507205 12299 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 19:07:47.507301 12299 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0406 19:07:47.507280 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 19:07:47.507360 12299 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0406 19:07:47.507447 12299 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 19:07:47.507512 12299 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 19:07:47.507594 12299 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 19:07:47.507756 12299 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 19:07:47.507773 12299 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0406 19:07:47.507917 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 19:07:47.507935 12299 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 19:07:47.507992 12299 iamrole.go:182] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0406 19:07:47.508010 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 19:07:47.508047 12299 iamrole.go:139] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0406 19:07:47.508104 12299 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 19:07:47.508151 12299 iamrole.go:182] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0406 19:07:47.508164 12299 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 19:07:47.508193 12299 iamrole.go:139] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0406 19:07:47.508258 12299 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 19:07:47.508338 12299 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 19:07:47.508412 12299 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 19:07:47.508680 12299 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.43s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 52.300s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.014s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0406 19:06:33.271589 6983 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.066s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.016s === RUN TestPodController I0406 19:06:36.299796 8319 pod.go:59] starting pod controller I0406 19:06:36.300080 8319 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.01s) I0406 19:06:36.303163 8319 pod.go:107] Got stop signal I0406 19:06:36.304343 8319 pod.go:65] shutting down pod controller PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.039s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.017s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000111d70} route53_test.go:93: Got zone list: [0xc000111da0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132080} --- PASS: TestZonesList (0.01s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000111e70} route53_test.go:93: Got zone list: [0xc000111ea0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc000111f40} route53_test.go:170: Successfully added managed DNS zone: &{0xc000436660 0xc0001320a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044e0a0} route53_test.go:93: Got zone list: [0xc00044e0d0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc0001320c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044e1d0} route53_test.go:93: Got zone list: [0xc00044e200] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc0001320e0} route53_test.go:187: Successfully added resource record set: {0xc00021a100 0xc000132100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044e480} route53_test.go:93: Got zone list: [0xc00044e4b0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132188} route53_test.go:199: Successfully added resource record set: {0xc00021a280 0xc0001321a8} route53_test.go:118: Got 1 recordsets: [0xc00044e650] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044e780} route53_test.go:93: Got zone list: [0xc00044e7b0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132240} route53_test.go:221: Successfully added resource record set: {0xc00021a400 0xc000132260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc00021a400 0xc000132260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044eb10} route53_test.go:93: Got zone list: [0xc00044eb40] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132318} route53_test.go:246: Successfully removed resource set {0xc00021a600 0xc000132338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044edc0} route53_test.go:93: Got zone list: [0xc00044edf0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc0001323c0} route53_test.go:264: Successfully removed resource set {0xc00021a780 0xc0001323e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044f0a0} route53_test.go:93: Got zone list: [0xc00044f0d0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132470} commontests.go:192: Successfully added resource record set: {0xc00021a900 0xc000132490} commontests.go:45: Correctly replaced resource record {0xc00021a900 0xc000132490} -> {0xc00021aa00 0xc0001324e0} commontests.go:142: Got 1 recordsets: [0xc00044f3e0] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044f5b0} route53_test.go:93: Got zone list: [0xc00044f5e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc000132590} commontests.go:192: Successfully added resource record set: {0xc00021ac80 0xc0001325b0} commontests.go:70: Correctly replaced resource record {0xc00021ac80 0xc0001325b0} -> {0xc00021ad80 0xc000132600} commontests.go:142: Got 1 recordsets: [0xc00044f8f0] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00044fb10} route53_test.go:93: Got zone list: [0xc00044fb40] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc0001326c0} commontests.go:192: Successfully added resource record set: {0xc00021b000 0xc0001326e0} commontests.go:142: Got 2 recordsets: [0xc00044fdb0 0xc00044fdd0] commontests.go:142: Got 2 recordsets: [0xc00044fe40 0xc00044fe60] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00045e000} route53_test.go:93: Got zone list: [0xc00045e030] route53_test.go:98: Got at least 1 zone in list:&{0xc000436480 0xc0001327e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.063s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.035s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc0006480a0} clouddns_test.go:71: Got zone list: [0xc0005a1c20] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc0005a1bf0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc0006481a0} clouddns_test.go:71: Got zone list: [0xc0005a1cc8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc0005a1c98} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc0006482a0} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00064c8c0 0xc0005a1d70} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648360} clouddns_test.go:71: Got zone list: [0xc0005a1e18] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc0005a1de8} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648480} clouddns_test.go:71: Got zone list: [0xc0005a1f38] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc0005a1f08} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc0006485c0} clouddns_test.go:71: Got zone list: [0xc00000e0a8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000e078} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648720} clouddns_test.go:71: Got zone list: [0xc00000e2a0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000e270} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648880} clouddns_test.go:71: Got zone list: [0xc00000e420] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000e3f0} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc0006489c0} clouddns_test.go:71: Got zone list: [0xc00000e570] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000e540} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648b20} clouddns_test.go:71: Got zone list: [0xc00000e720] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000e6f0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648c80} clouddns_test.go:71: Got zone list: [0xc00000e9d8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000e9a8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648e00} clouddns_test.go:71: Got zone list: [0xc00000ecf0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000ecc0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000597688 0xc000648fe0} clouddns_test.go:71: Got zone list: [0xc00000f0b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00064c780 0xc00000f080} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.090s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.057s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.141s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0406 19:06:50.610330 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.610398 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.610429 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.610469 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.610512 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.610527 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.610547 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.610571 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.610584 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.618685 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0406 19:06:50.622623 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.622657 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.622689 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.622720 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.622747 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.622764 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.622793 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.622826 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.622852 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.624182 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0406 19:06:50.627816 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.627854 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.627893 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.627936 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.627978 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.628004 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.628046 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.628098 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.628133 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.629147 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.630616 12072 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0406 19:06:50.630638 12072 assetstore.go:111] bin/containerd I0406 19:06:50.630651 12072 assetstore.go:111] bin/containerd-shim I0406 19:06:50.630663 12072 assetstore.go:111] bin/containerd-shim-runc-v1 I0406 19:06:50.630673 12072 assetstore.go:111] bin/containerd-shim-runc-v2 I0406 19:06:50.630685 12072 assetstore.go:111] bin/containerd-stress I0406 19:06:50.630696 12072 assetstore.go:111] bin/ctr I0406 19:06:50.630748 12072 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0406 19:06:50.630769 12072 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0406 19:06:50.633942 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.633977 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.634014 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.634056 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.634088 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.634117 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.634147 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.634177 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.634201 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.635202 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.636547 12072 containerd.go:57] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0406 19:06:50.637560 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.637591 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.637629 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.637667 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.637755 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.637802 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.637832 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.637859 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.637882 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.638832 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.640236 12072 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0406 19:06:50.641060 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.641095 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.641131 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.641170 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.641220 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.641250 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.641280 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.641307 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.641330 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.642311 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.643727 12072 assetstore.go:108] Matching assets for "^docker/": I0406 19:06:50.643747 12072 assetstore.go:111] docker/containerd I0406 19:06:50.643760 12072 assetstore.go:111] docker/containerd-shim I0406 19:06:50.643771 12072 assetstore.go:111] docker/ctr I0406 19:06:50.643782 12072 assetstore.go:111] docker/docker I0406 19:06:50.643792 12072 assetstore.go:111] docker/docker-init I0406 19:06:50.643804 12072 assetstore.go:111] docker/docker-proxy I0406 19:06:50.643815 12072 assetstore.go:111] docker/dockerd I0406 19:06:50.643825 12072 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0406 19:06:50.646712 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.646756 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.646793 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.646839 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.646891 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.646921 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.646950 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.646978 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.647002 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.647895 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.649858 12072 assetstore.go:108] Matching assets for "^docker/": I0406 19:06:50.649879 12072 assetstore.go:111] docker/containerd I0406 19:06:50.649891 12072 assetstore.go:111] docker/containerd-shim I0406 19:06:50.649902 12072 assetstore.go:111] docker/ctr I0406 19:06:50.649912 12072 assetstore.go:111] docker/docker I0406 19:06:50.649923 12072 assetstore.go:111] docker/docker-init I0406 19:06:50.649934 12072 assetstore.go:111] docker/docker-proxy I0406 19:06:50.649944 12072 assetstore.go:111] docker/dockerd I0406 19:06:50.649954 12072 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0406 19:06:50.654112 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.654150 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.654186 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.654227 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.654280 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.654311 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.654341 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.654370 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.654393 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.655388 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.656755 12072 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0406 19:06:50.657031 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.657061 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.657097 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.657136 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.657184 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.657212 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.657240 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.657269 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.657291 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.658265 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:50.659614 12072 assetstore.go:108] Matching assets for "^docker/": I0406 19:06:50.659635 12072 assetstore.go:111] docker/docker I0406 19:06:50.659647 12072 assetstore.go:111] docker/docker-containerd I0406 19:06:50.659659 12072 assetstore.go:111] docker/docker-containerd-ctr I0406 19:06:50.659672 12072 assetstore.go:111] docker/docker-containerd-shim I0406 19:06:50.659682 12072 assetstore.go:111] docker/docker-init I0406 19:06:50.659693 12072 assetstore.go:111] docker/docker-proxy I0406 19:06:50.659704 12072 assetstore.go:111] docker/docker-runc I0406 19:06:50.659714 12072 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0406 19:06:50.663786 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.663834 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.663869 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.663913 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.663942 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.663970 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.663999 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.664028 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.664052 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.666292 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0406 19:06:50.668662 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.668706 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.668744 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.668787 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.668822 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.668852 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.668883 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.668912 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.668938 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.670568 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0406 19:06:50.673006 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.673054 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.673093 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.673139 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.673171 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.673203 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.673236 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.673267 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.673291 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.674836 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0406 19:06:50.680703 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.680753 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.680793 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.680836 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.680868 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.680903 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.680935 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.680966 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.680991 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.682573 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0406 19:06:50.693001 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.693061 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.693092 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.693125 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.693151 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.693167 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.693184 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.693204 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.693220 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.694894 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0406 19:06:50.700878 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.700921 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.700945 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.700972 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.700994 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.701012 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.701029 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.701046 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.701060 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.702506 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0406 19:06:50.708821 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.708875 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.708903 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.708955 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.708986 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.709005 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.709022 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.709043 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.709059 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.710836 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.02s) === RUN TestKubeAPIServerBuilderAMD64 I0406 19:06:50.729526 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.729619 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:50.729670 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:50.729723 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:50.729768 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:50.729796 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:50.729828 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:50.729860 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:50.729887 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:50.732890 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (0.57s) === RUN TestKubeAPIServerBuilderARM64 I0406 19:06:51.295159 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.295230 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.295276 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.295318 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.295350 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.295373 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.295405 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.295431 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.295453 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.297689 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0406 19:06:51.373326 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.373376 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.373402 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.373427 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.373450 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.373463 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.373478 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.373495 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.373508 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.375039 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0406 19:06:51.380128 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.380172 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.380195 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.380220 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.380243 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.380258 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.380273 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.380290 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.380303 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.381785 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0406 19:06:51.449148 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.449195 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.449225 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.449251 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.449272 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.449285 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.449299 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.449319 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.449333 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.450820 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0406 19:06:51.517018 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.517060 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.517085 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.517109 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.517132 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.517145 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.517159 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.517180 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.517193 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.518641 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.00s) === RUN TestKubeProxyBuilderAMD64 I0406 19:06:51.521608 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.521642 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.521663 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.521684 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.521703 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.521717 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.521732 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.521746 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.521757 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.523070 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.06s) === RUN TestKubeProxyBuilderARM64 I0406 19:06:51.586017 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.586063 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.586092 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.586117 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.586140 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.586153 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.586170 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.586188 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.586201 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.587698 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0406 19:06:51.651759 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.651812 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.651841 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.651873 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.651900 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.651917 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.651935 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.651956 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.651971 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.654058 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0406 19:06:51.659809 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.659863 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.659895 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.659926 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.659952 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.659969 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.659987 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.660007 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.660023 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.661698 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0406 19:06:51.726616 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.726670 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.726729 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.726762 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.726788 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.726809 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.726828 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.726849 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.726864 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.728475 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0406 19:06:51.791655 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.791695 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.791718 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.791743 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.791762 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.791775 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.791790 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.791806 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.791817 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.793100 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:51.794485 12072 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.00s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0406 19:06:51.797509 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.797552 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.797581 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.797608 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.797631 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.797647 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.797662 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.797678 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.797691 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.799009 12072 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0406 19:06:51.802681 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.802723 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.802747 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.802773 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.802796 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.802812 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.802831 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.802857 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.802872 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.804159 12072 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0406 19:06:51.807843 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.807887 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.807914 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.807943 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.807968 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.807986 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.808006 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.808025 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.808040 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.809688 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:51.811274 12072 assetstore.go:108] Matching assets for "protokube$": I0406 19:06:51.811293 12072 assetstore.go:111] /opt/kops/bin/protokube I0406 19:06:51.811303 12072 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0406 19:06:51.811322 12072 assetstore.go:108] Matching assets for "channels$": I0406 19:06:51.811332 12072 assetstore.go:111] /opt/kops/bin/channels I0406 19:06:51.811338 12072 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0406 19:06:51.812873 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.812914 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.812944 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.812971 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.813004 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.813020 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.813037 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.813054 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.813069 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.814575 12072 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0406 19:06:51.816768 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.816807 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.816831 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.816855 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.816878 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.816894 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.816912 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.816927 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.816942 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.818421 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:51.820103 12072 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0406 19:06:51.820570 12072 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:51.820607 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:51.820632 12072 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:51.820657 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:51.820678 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:51.820695 12072 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:51.820714 12072 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:51.820730 12072 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:51.820744 12072 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:51.822219 12072 zones.go:93] ListHostedZonesPages { } I0406 19:06:51.823754 12072 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 1.266s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.051s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.083s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.082s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.070s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0406 19:06:34.582560 7694 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0406 19:06:34.582647 7694 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.057s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0406 19:06:43.342567 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.342679 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.342719 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.342749 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.342779 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0406 19:06:43.342937 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.342976 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343004 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343033 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343065 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343092 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343125 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343154 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343189 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343221 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343246 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343280 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343322 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343352 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343376 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343432 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343463 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343489 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 19:06:43.343513 10447 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0406 19:06:43.343635 10447 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0406 19:06:43.343709 10447 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvdia --- PASS: Test_Validate_Nvdia (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.172s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0406 19:06:35.558899 8090 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0406 19:06:35.558952 8090 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0406 19:06:35.558961 8090 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0406 19:06:35.558988 8090 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.129s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.015s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.161s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.097s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.01s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.240s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.075s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.032s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.023s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.130s === RUN TestFlagToFalse I0406 19:06:34.878757 7899 featureflag_test.go:33] Created flag Unittest1 I0406 19:06:34.878820 7899 featureflag.go:162] FeatureFlag "UnitTest1"=false I0406 19:06:34.878842 7899 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0406 19:06:34.878926 7899 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.030s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.073s === RUN TestRenderInstanceGroupZones W0406 19:06:37.953753 8847 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.068s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0406 19:07:03.456953 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.457014 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.458172 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.458198 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.463410 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.463433 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.463445 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.463453 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.465850 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.466756 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.466794 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.466818 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.468015 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.468050 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.473328 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.473378 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:03.476331 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:03.476364 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.476387 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.477565 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.477599 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.482843 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.482898 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.482923 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.482950 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.484068 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.484950 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.484984 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.485007 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.486163 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.486209 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.491487 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.491716 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.492607 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.492638 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.492661 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.493911 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.493952 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.499213 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.499264 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.500121 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.500153 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.500174 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.501385 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.501432 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.506710 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.506744 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.506764 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.506778 12353 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0406 19:07:03.507832 12353 instancegroups.go:398] Draining the node: "node-2a.local". I0406 19:07:03.508622 12353 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0406 19:07:03.508655 12353 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 19:07:03.508678 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.509855 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.509889 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.515118 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.515175 12353 instancegroups.go:398] Draining the node: "node-2b.local". I0406 19:07:03.516064 12353 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0406 19:07:03.516096 12353 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 19:07:03.516118 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.517320 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.517358 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.522602 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.522651 12353 instancegroups.go:398] Draining the node: "node-2c.local". I0406 19:07:03.523514 12353 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0406 19:07:03.523546 12353 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 19:07:03.523570 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.524785 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.524820 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.530081 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.530116 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0406 19:07:03.530765 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.530799 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.531963 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.532024 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.532068 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.532084 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.532109 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.533310 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.533360 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.533375 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.533402 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.534598 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.534631 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.534678 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.534692 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.534716 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.535912 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.535963 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.535978 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.536004 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.537196 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.537245 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.537259 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.537322 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.538510 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.538537 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.538588 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.538602 12353 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 19:07:03.538629 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.539820 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.539869 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.539887 12353 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 19:07:03.539913 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.541101 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0406 19:07:03.541151 12353 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0406 19:07:03.541165 12353 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 19:07:03.541191 12353 instancegroups.go:435] waiting for 1ms after terminating instance W0406 19:07:03.542384 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0406 19:07:03.542416 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0406 19:07:03.542700 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.542730 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.543929 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.543970 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.543991 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.544047 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.544070 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.544085 12353 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.544100 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.544973 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.545864 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.545896 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.545920 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.547112 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.547151 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.547173 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.547213 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:03.548046 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:03.548102 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.548129 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.549331 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.549370 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.549391 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.549411 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.549430 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.549444 12353 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.549458 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.550500 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.551308 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.551342 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.551367 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.552576 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.552615 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.552636 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.552675 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.553594 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.553626 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.553648 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.554806 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.554846 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.554866 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.554906 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.555822 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.555854 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.555876 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.557068 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.557106 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.557127 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.557142 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.557159 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.557172 12353 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.557186 12353 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0406 19:07:03.558244 12353 instancegroups.go:398] Draining the node: "node-2a.local". I0406 19:07:03.559016 12353 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0406 19:07:03.559048 12353 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 19:07:03.559070 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.560271 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.560326 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.560351 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.560430 12353 instancegroups.go:398] Draining the node: "node-2b.local". I0406 19:07:03.561782 12353 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0406 19:07:03.561826 12353 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 19:07:03.561852 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.563059 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.563099 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.563120 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.563162 12353 instancegroups.go:398] Draining the node: "node-2c.local". I0406 19:07:03.564029 12353 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0406 19:07:03.564061 12353 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 19:07:03.564084 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.565207 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.565236 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0406 19:07:03.565254 12353 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 19:07:03.565270 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0406 19:07:03.565571 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0406 19:07:03.565816 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.565844 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.567032 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.567071 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.572331 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.572393 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.572415 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.572428 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.573193 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.573975 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.574007 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.574030 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.575218 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.575252 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.580483 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.580553 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:03.581423 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:03.581457 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.581481 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.582676 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.582710 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.587944 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.587983 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.588002 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.588016 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.589022 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.590218 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.590250 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.590274 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.591481 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.591522 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.596810 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.596859 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.597685 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.597717 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.597740 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.598933 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.598965 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.604198 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.604245 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.605042 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.605069 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.605088 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.606301 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.606340 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.611580 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.611611 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.611631 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.611684 12353 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0406 19:07:03.612628 12353 instancegroups.go:398] Draining the node: "node-2a.local". I0406 19:07:03.613388 12353 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0406 19:07:03.613420 12353 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 19:07:03.613444 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.614639 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.614673 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.619910 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.619960 12353 instancegroups.go:398] Draining the node: "node-2b.local". I0406 19:07:03.620788 12353 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0406 19:07:03.620819 12353 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 19:07:03.620842 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.622049 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.622094 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.627382 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.627432 12353 instancegroups.go:398] Draining the node: "node-2c.local". I0406 19:07:03.628304 12353 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0406 19:07:03.628334 12353 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 19:07:03.628356 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.629565 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.629609 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.634877 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.634911 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0406 19:07:03.635064 12353 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0406 19:07:03.635527 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.635557 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.636670 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.636717 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.636740 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0406 19:07:03.637193 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.637225 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.638365 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.638408 12353 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0406 19:07:03.638431 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0406 19:07:03.638680 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.638708 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.638723 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0406 19:07:03.638895 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.638919 12353 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0406 19:07:03.638935 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0406 19:07:03.639161 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.639189 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.640306 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.640335 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.645572 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.645631 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.645655 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.645668 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.646410 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.647157 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.647205 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.647230 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.648416 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.648462 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.648483 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0406 19:07:03.648827 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.648858 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.650051 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.650090 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.655335 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.655397 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.655421 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.655434 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.656203 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.656963 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.656993 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.657017 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.658203 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.658246 12353 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0406 19:07:03.658268 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0406 19:07:03.658556 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.658582 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.658595 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.659597 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.660395 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.660425 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.660448 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.661631 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.661670 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.667343 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.667394 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.668238 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.668270 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.668293 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.669491 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.669535 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.674790 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.674840 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.675646 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.675677 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.675699 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.676900 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.676935 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.682192 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.682226 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0406 19:07:03.682550 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.682579 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.682600 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0406 19:07:03.682858 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.682884 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.682899 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0406 19:07:03.683147 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.683175 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.683190 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0406 19:07:03.683347 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.683374 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.683386 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.684375 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.685157 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.685188 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.685213 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.686354 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.686396 12353 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0406 19:07:03.686416 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0406 19:07:03.686655 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.686687 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.686700 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.687707 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.688508 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.688539 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.688560 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.689754 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.689799 12353 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0406 19:07:03.689821 12353 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0406 19:07:03.690118 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.690159 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.691347 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.691442 12353 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 19:07:03.692608 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.697875 12353 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 19:07:03.699048 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.704365 12353 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 19:07:03.705658 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.710894 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.710984 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.711026 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.711043 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.712093 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.713969 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.714004 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.714031 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.715221 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.715284 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.720592 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.720644 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:03.721581 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:03.721614 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.721637 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.722835 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.722891 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.728161 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.728201 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.728228 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.728242 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.729229 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.730003 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.730039 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.730065 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.731267 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.731314 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.736612 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.736682 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.737515 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.737549 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.737571 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.738776 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.738825 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.744113 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.744197 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.745182 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.745215 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.745239 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.746442 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.746489 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.751747 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.751779 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.751806 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.751821 12353 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0406 19:07:03.752764 12353 instancegroups.go:398] Draining the node: "node-2a.local". I0406 19:07:03.753546 12353 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0406 19:07:03.753577 12353 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 19:07:03.753599 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.754796 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.754853 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.760125 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.760184 12353 instancegroups.go:398] Draining the node: "node-2b.local". I0406 19:07:03.760973 12353 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0406 19:07:03.761004 12353 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 19:07:03.761026 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.762238 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.762287 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.767556 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.767618 12353 instancegroups.go:398] Draining the node: "node-2c.local". I0406 19:07:03.768469 12353 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0406 19:07:03.768502 12353 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 19:07:03.768524 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.769725 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.769774 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.775062 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.775097 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I0406 19:07:03.775416 12353 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 19:07:03.775445 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.776637 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.776678 12353 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 19:07:03.777837 12353 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 19:07:03.779024 12353 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 19:07:03.780161 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.785379 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.785441 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.785472 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.785487 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.786180 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.786963 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.786994 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.787017 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.788205 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.788241 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.793506 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.793558 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:03.794375 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:03.794407 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.794431 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.795631 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.795666 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.800898 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.800937 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.800958 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.800972 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.802320 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.803299 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.803334 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.803357 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.804555 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.804589 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.809801 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.809853 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.810667 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.810701 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.810724 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.811916 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.811959 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.817183 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.817235 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.818063 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.818117 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.818142 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.819334 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.819368 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.824613 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.824645 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.824666 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.824681 12353 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0406 19:07:03.825608 12353 instancegroups.go:398] Draining the node: "node-2a.local". I0406 19:07:03.826367 12353 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0406 19:07:03.826398 12353 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 19:07:03.826420 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.827613 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.827647 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.832875 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.832927 12353 instancegroups.go:398] Draining the node: "node-2b.local". I0406 19:07:03.837074 12353 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0406 19:07:03.837119 12353 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 19:07:03.837144 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.838340 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.838377 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.843615 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.843670 12353 instancegroups.go:398] Draining the node: "node-2c.local". I0406 19:07:03.844462 12353 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0406 19:07:03.844493 12353 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 19:07:03.844516 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.845675 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.845717 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.850992 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.851028 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.08s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0406 19:07:03.851764 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.851791 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.851806 12353 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0406 19:07:03.852519 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.853294 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.853325 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.853347 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.854548 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.854583 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.859830 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.859887 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.860714 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.860763 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.860790 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.861973 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.862009 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.867214 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.867274 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0406 19:07:03.867680 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.867714 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.867729 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.868619 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:03.869523 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:03.869562 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:03.869589 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.870812 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.870848 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.876111 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.876164 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:03.876994 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:03.877029 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:03.877052 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.878252 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.878287 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.883540 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.883574 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0406 19:07:03.883893 12353 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled W0406 19:07:03.883936 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0406 19:07:03.883955 12353 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled W0406 19:07:03.883970 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0406 19:07:03.883988 12353 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled W0406 19:07:03.883997 12353 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0406 19:07:03.884014 12353 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0406 19:07:03.884029 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0406 19:07:03.884266 12353 instancegroups.go:568] Detaching instance "bastion-1a", in group "bastion-1". I0406 19:07:03.884320 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:03.885525 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.885560 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.890806 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.890848 12353 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled I0406 19:07:03.890899 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.890919 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.890932 12353 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0406 19:07:03.892008 12353 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled I0406 19:07:03.892053 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.892084 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.892104 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:03.893688 12353 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0406 19:07:03.893781 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:03.894991 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.895026 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.900315 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.900353 12353 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled I0406 19:07:03.900369 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.900388 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.900401 12353 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0406 19:07:03.901348 12353 instancegroups.go:566] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0406 19:07:03.901419 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:03.902620 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.902655 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:03.907875 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.907910 12353 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0406 19:07:03.907929 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0406 19:07:03.908222 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.908259 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.908274 12353 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0406 19:07:03.910357 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:03.911126 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:03.911160 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:03.911192 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.912410 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.912458 12353 instancegroups.go:501] Cluster validated. I0406 19:07:03.912511 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:03.912584 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:03.914016 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:03.914051 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:03.914392 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:03.914428 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:03.914460 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:03.915671 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:03.915747 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.016393 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.016480 12353 instancegroups.go:398] Draining the node: "node-1e.local". I0406 19:07:04.016571 12353 instancegroups.go:398] Draining the node: "node-1d.local". I0406 19:07:04.017966 12353 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0406 19:07:04.018004 12353 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 19:07:04.018409 12353 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0406 19:07:04.018454 12353 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 19:07:04.018494 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.019715 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.019764 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.019807 12353 instancegroups.go:398] Draining the node: "node-1f.local". I0406 19:07:04.020623 12353 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0406 19:07:04.020654 12353 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 19:07:04.020686 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.021886 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.021963 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.122597 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.122652 12353 instancegroups.go:398] Draining the node: "node-1g.local". I0406 19:07:04.123502 12353 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0406 19:07:04.123537 12353 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0406 19:07:04.123569 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.124813 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.124856 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.124874 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0406 19:07:04.125119 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.125151 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.125166 12353 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0406 19:07:04.126926 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:04.126961 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:04.128132 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:04.128167 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:04.128488 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:04.128525 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:04.128559 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.129790 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.129851 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.230422 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.230471 12353 instancegroups.go:398] Draining the node: "node-1d.local". I0406 19:07:04.230505 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:04.231822 12353 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0406 19:07:04.231857 12353 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 19:07:04.232125 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:04.232154 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:04.232182 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.233401 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.233444 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.233482 12353 instancegroups.go:398] Draining the node: "node-1e.local". I0406 19:07:04.234290 12353 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0406 19:07:04.234323 12353 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 19:07:04.234354 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.235560 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.235619 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.336175 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.336223 12353 instancegroups.go:398] Draining the node: "node-1f.local". I0406 19:07:04.337048 12353 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0406 19:07:04.337079 12353 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 19:07:04.337110 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.338294 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.338337 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.338354 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0406 19:07:04.338557 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.338590 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.338604 12353 instancegroups.go:309] Tainting 7 nodes in "master-1" instancegroup. I0406 19:07:04.341032 12353 instancegroups.go:398] Draining the node: "master-1a.local". I0406 19:07:04.341803 12353 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0406 19:07:04.341835 12353 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 19:07:04.341868 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.343055 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.343098 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.343137 12353 instancegroups.go:398] Draining the node: "master-1c.local". I0406 19:07:04.343219 12353 instancegroups.go:398] Draining the node: "master-1b.local". I0406 19:07:04.343979 12353 instancegroups.go:417] deleting node "master-1c.local" from kubernetes I0406 19:07:04.344011 12353 instancegroups.go:589] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0406 19:07:04.344608 12353 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0406 19:07:04.344638 12353 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 19:07:04.344668 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.345881 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.345952 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.446545 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.446597 12353 instancegroups.go:398] Draining the node: "master-1e.local". I0406 19:07:04.446684 12353 instancegroups.go:398] Draining the node: "master-1d.local". I0406 19:07:04.447957 12353 instancegroups.go:417] deleting node "master-1e.local" from kubernetes I0406 19:07:04.447997 12353 instancegroups.go:589] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0406 19:07:04.448342 12353 instancegroups.go:417] deleting node "master-1d.local" from kubernetes I0406 19:07:04.448379 12353 instancegroups.go:589] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0406 19:07:04.448411 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.449616 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.449659 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.449699 12353 instancegroups.go:398] Draining the node: "master-1f.local". I0406 19:07:04.450514 12353 instancegroups.go:417] deleting node "master-1f.local" from kubernetes I0406 19:07:04.450546 12353 instancegroups.go:589] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0406 19:07:04.450577 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.451771 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.451838 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.552455 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.552503 12353 instancegroups.go:398] Draining the node: "master-1g.local". I0406 19:07:04.553319 12353 instancegroups.go:417] deleting node "master-1g.local" from kubernetes I0406 19:07:04.553354 12353 instancegroups.go:589] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0406 19:07:04.553385 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.554595 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.554638 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.554655 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0406 19:07:04.554903 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.554940 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.554954 12353 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0406 19:07:04.556698 12353 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0406 19:07:04.556763 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:04.557967 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.558010 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.558023 12353 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0406 19:07:04.558092 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:04.559298 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.559342 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.559386 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:04.559444 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:04.560779 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:04.561106 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:04.561141 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:04.561217 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:04.561267 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.562476 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.562546 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.663125 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.663174 12353 instancegroups.go:398] Draining the node: "node-1d.local". I0406 19:07:04.663208 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:04.664514 12353 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0406 19:07:04.664549 12353 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 19:07:04.664843 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:04.664873 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:04.664902 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.666120 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.666164 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.666204 12353 instancegroups.go:398] Draining the node: "node-1e.local". I0406 19:07:04.667025 12353 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0406 19:07:04.667057 12353 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 19:07:04.667090 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.668284 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.668344 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.768889 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.768937 12353 instancegroups.go:398] Draining the node: "node-1f.local". I0406 19:07:04.769778 12353 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0406 19:07:04.769810 12353 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 19:07:04.769844 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.771041 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.771084 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.771101 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0406 19:07:04.771306 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.771337 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.771351 12353 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0406 19:07:04.775064 12353 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0406 19:07:04.775120 12353 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0406 19:07:04.775149 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:04.776525 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.776574 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.776626 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:04.776724 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:04.777922 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:04.777963 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:04.778328 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:04.778365 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:04.778397 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.779585 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.779666 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.880264 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.880326 12353 instancegroups.go:398] Draining the node: "node-1d.local". I0406 19:07:04.880379 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:04.881795 12353 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0406 19:07:04.881836 12353 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 19:07:04.882193 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:04.882234 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:04.882271 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.883503 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.883552 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.883594 12353 instancegroups.go:398] Draining the node: "node-1e.local". I0406 19:07:04.884427 12353 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0406 19:07:04.884462 12353 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 19:07:04.884497 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.885701 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.885768 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.986805 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.986856 12353 instancegroups.go:398] Draining the node: "node-1f.local". I0406 19:07:04.987703 12353 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0406 19:07:04.987737 12353 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 19:07:04.987774 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.989041 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.989113 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.989134 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0406 19:07:04.989443 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.989469 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.989483 12353 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0406 19:07:04.990251 12353 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0406 19:07:04.990289 12353 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0406 19:07:04.990324 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:04.991531 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.991568 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:04.996833 12353 instancegroups.go:501] Cluster validated. I0406 19:07:04.996904 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:04.996926 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:04.998266 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:04.998314 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:04.998347 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.998730 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:04.998772 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:04.998799 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:04.999511 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:04.999547 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:05.004932 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.004965 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0406 19:07:05.005191 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.005218 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.005230 12353 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0406 19:07:05.005921 12353 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0406 19:07:05.005961 12353 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0406 19:07:05.006000 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:05.007144 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.007179 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:05.012437 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.012492 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:05.012529 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:05.013868 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:05.013905 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:05.013931 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.014282 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:05.014322 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:05.014347 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.015131 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.015165 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:05.020745 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.020782 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0406 19:07:05.021088 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.021125 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.021140 12353 instancegroups.go:309] Tainting 4 nodes in "node-1" instancegroup. I0406 19:07:05.022458 12353 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0406 19:07:05.022522 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:05.024087 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.024130 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.024145 12353 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0406 19:07:05.024198 12353 instancegroups.go:170] waiting for 1ms after detaching instance I0406 19:07:05.025361 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.025413 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.025476 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:05.025551 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:05.025619 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:05.027319 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:05.027356 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:05.027392 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.027712 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:05.027751 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:05.027786 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.028055 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:05.028115 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:05.028156 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.028599 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.129135 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.129192 12353 instancegroups.go:398] Draining the node: "node-1d.local". I0406 19:07:05.130044 12353 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0406 19:07:05.130079 12353 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 19:07:05.130114 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.131371 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.131436 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.131458 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0406 19:07:05.131703 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.131741 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.131755 12353 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0406 19:07:05.133887 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:05.134698 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:05.134734 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:05.134797 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.136016 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.136060 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.136106 12353 instancegroups.go:398] Draining the node: "node-1e.local". I0406 19:07:05.136187 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:05.137457 12353 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0406 19:07:05.137494 12353 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 19:07:05.137834 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:05.137872 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:05.137905 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.139069 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.139134 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.240180 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.240235 12353 instancegroups.go:398] Draining the node: "node-1g.local". I0406 19:07:05.240270 12353 instancegroups.go:398] Draining the node: "node-1f.local". I0406 19:07:05.241626 12353 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0406 19:07:05.241665 12353 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0406 19:07:05.242011 12353 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0406 19:07:05.242050 12353 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 19:07:05.242083 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.243295 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.243338 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.243378 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:05.244246 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:05.244284 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:05.244315 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.245532 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.245597 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.346352 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.346402 12353 instancegroups.go:398] Draining the node: "node-1d.local". I0406 19:07:05.347351 12353 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0406 19:07:05.347387 12353 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 19:07:05.347419 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.348662 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.348708 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.348726 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0406 19:07:05.348932 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.348958 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.348979 12353 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0406 19:07:05.349002 12353 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0406 19:07:05.349020 12353 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0406 19:07:05.349036 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0406 19:07:05.349181 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.349205 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.349218 12353 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0406 19:07:05.350215 12353 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0406 19:07:05.350247 12353 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0406 19:07:05.350266 12353 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0406 19:07:05.350305 12353 instancegroups.go:398] Draining the node: "node-1a.local". I0406 19:07:05.351102 12353 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0406 19:07:05.351133 12353 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 19:07:05.351154 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.352347 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.352384 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:05.357683 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.357733 12353 instancegroups.go:398] Draining the node: "node-1b.local". I0406 19:07:05.358607 12353 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0406 19:07:05.358638 12353 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 19:07:05.358661 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.359886 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.359922 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:05.365170 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.365221 12353 instancegroups.go:398] Draining the node: "node-1c.local". I0406 19:07:05.366047 12353 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0406 19:07:05.366082 12353 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 19:07:05.366104 12353 instancegroups.go:435] waiting for 1ms after terminating instance I0406 19:07:05.367325 12353 instancegroups.go:468] Validating the cluster. I0406 19:07:05.367361 12353 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 19:07:05.372577 12353 instancegroups.go:501] Cluster validated. I0406 19:07:05.372609 12353 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.992s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.009s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.040s === RUN TestParse E0406 19:06:36.503710 8415 versions.go:96] unable to parse Kubernetes version "" E0406 19:06:36.503783 8415 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.018s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.034s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.15s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.306s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.095s ? k8s.io/kops/pkg/model/alimodel [no test files] === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.075s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.098s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.183s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/pollinterval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0406 19:06:47.736313 11715 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0406 19:06:47.736427 11715 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0406 19:06:47.737410 11715 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0406 19:06:47.737522 11715 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/pollinterval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.105s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.149s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.025s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.147s === RUN TestServerGroupBuilder --- PASS: TestServerGroupBuilder (0.25s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.312s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.057s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.042s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.113s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (2.03s) --- PASS: TestIssueCert/ca (0.67s) --- PASS: TestIssueCert/client (0.32s) --- PASS: TestIssueCert/clientOneYear (1.03s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTrip --- PASS: TestPrivateKeyRoundTrip (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 2.057s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.018s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/ali [no test files] === RUN TestAddUntaggedRouteTables I0406 19:06:49.221111 11963 routetable.go:53] DescribeRouteTables: { } W0406 19:06:49.221236 11963 tags.go:67] cluster tag not found on route-table:rtb-1234 I0406 19:06:49.221253 11963 aws.go:224] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0406 19:06:49.221358 11963 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0406 19:06:49.221469 11963 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0406 19:06:49.221564 11963 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0406 19:06:49.221666 11963 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0406 19:06:49.221703 11963 volumes.go:128] DescribeVolumes: { } W0406 19:06:49.221739 11963 aws.go:2236] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0406 19:06:49.222037 11963 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:49.222079 11963 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:49.222145 11963 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:49.222210 11963 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:49.222269 11963 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:49.222299 11963 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:49.222344 11963 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:49.222380 11963 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:49.222402 11963 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:49.222441 11963 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0406 19:06:49.222472 11963 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0406 19:06:49.222505 11963 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0406 19:06:49.222541 11963 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0406 19:06:49.222568 11963 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0406 19:06:49.222605 11963 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0406 19:06:49.222647 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0406 19:06:49.222684 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0406 19:06:49.222724 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0406 19:06:49.222753 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0406 19:06:49.222781 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0406 19:06:49.222805 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0406 19:06:49.222840 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0406 19:06:49.222863 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0406 19:06:49.222891 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0406 19:06:49.222925 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0406 19:06:49.222953 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0406 19:06:49.222975 11963 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.067s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.083s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.145s ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.045s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.069s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.048s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.032s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.03s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.01s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.01s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.02s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.170s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.060s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/ali [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.045s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0406 19:06:40.309260 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309833 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309839 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309846 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309851 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309859 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309867 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309876 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309885 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309894 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309902 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309909 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309914 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309920 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.309926 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.309932 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312399 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312430 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312489 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312502 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312552 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312565 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312614 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312627 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312683 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312697 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312748 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312761 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312810 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312822 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312885 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312897 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.312946 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.312959 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313006 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313018 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313066 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313079 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313138 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313151 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313199 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313212 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313260 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313276 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313329 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313342 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313392 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313405 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313454 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313466 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313514 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313527 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313585 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313608 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313665 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313681 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313742 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313759 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313821 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313850 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313915 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.313931 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.313994 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314009 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314068 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314083 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314142 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314158 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314234 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314250 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314307 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314322 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314382 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314398 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314456 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314471 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314530 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314546 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314611 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314626 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314697 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314713 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314803 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314828 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314896 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314913 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.314974 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.314990 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315050 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315066 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315127 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315143 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315204 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315221 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315285 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315301 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315360 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315376 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315446 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315461 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315517 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315532 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315584 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315598 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315655 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315672 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315727 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315742 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315796 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315808 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315860 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315874 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.315965 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.315980 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316041 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316056 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316111 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316123 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316178 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316191 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316246 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316259 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316313 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316324 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316379 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316393 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316447 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316460 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316513 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316527 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316582 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316598 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316651 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316664 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316721 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316736 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316793 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316808 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316863 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316878 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.316931 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.316946 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317013 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317028 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317083 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317098 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317168 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317184 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317240 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317256 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317310 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317325 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317380 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317395 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317453 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317468 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317530 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317546 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317605 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317622 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317682 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317698 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317760 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317777 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317835 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317852 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.317913 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.317929 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318011 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318028 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318104 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318124 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318186 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318203 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318278 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318294 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318357 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318373 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318436 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318453 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318511 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318527 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318587 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318604 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318679 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318699 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318763 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318779 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318857 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318875 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.318934 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.318949 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319012 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319027 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319090 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319106 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319166 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319182 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319245 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319259 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319318 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319333 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319394 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319409 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319468 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319482 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319547 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319568 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319631 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319646 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319711 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319727 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.319786 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.319802 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0406 19:06:40.320411 9844 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0406 19:06:40.320424 9844 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 19:06:40.320435 9844 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0406 19:06:40.320444 9844 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0406 19:06:40.320453 9844 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 19:06:40.320463 9844 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 19:06:40.320471 9844 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 19:06:40.320501 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320515 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320526 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.320536 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320548 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.320558 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320569 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.320579 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320589 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.320600 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320612 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320622 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320648 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320662 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.320672 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.320683 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329209 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329249 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329327 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329342 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329411 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329425 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329494 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329509 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329581 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329596 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329668 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329683 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329754 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329768 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329835 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329849 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.329922 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.329947 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330006 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330062 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330145 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330211 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330306 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330342 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330429 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330498 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330613 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330657 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330755 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330790 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.330904 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.330940 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331029 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331072 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331162 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331198 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331292 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331340 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331444 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331491 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331589 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331628 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331729 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331763 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331854 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.331898 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.331988 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332022 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332147 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332183 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332269 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332342 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332440 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332477 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332575 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332609 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332697 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332740 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332830 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.332867 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.332957 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333000 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333112 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333159 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333250 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333285 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333373 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333440 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333525 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333560 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333647 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333684 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333792 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333827 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.333905 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.333941 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334022 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334058 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334137 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334181 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334271 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334319 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334403 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334437 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334524 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334560 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334675 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334711 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334809 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334850 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.334942 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.334977 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335057 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335092 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335170 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335203 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335312 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335347 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335428 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335471 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335578 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335616 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335696 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335737 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335817 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335856 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.335936 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.335974 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336054 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336091 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336172 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336207 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336307 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336348 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336429 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336465 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336543 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336579 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336676 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336715 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336796 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.336832 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.336964 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337010 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337093 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337131 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337224 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337261 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337343 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337379 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337459 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337495 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337574 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337632 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337718 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337754 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337835 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337873 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.337952 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.337990 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338083 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338122 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338204 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338240 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338323 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338359 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338440 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338475 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338556 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338592 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338674 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338711 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338790 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.338826 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.338911 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339212 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339274 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339291 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339379 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339397 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339454 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339470 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339526 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339542 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339599 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339615 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339674 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339690 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339749 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339767 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339827 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339843 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339906 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339922 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.339976 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.339992 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340044 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340061 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340180 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340198 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340253 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340271 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340331 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340347 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340399 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340413 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340473 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340490 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340577 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340593 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340673 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340688 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340748 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340763 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340820 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340835 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 19:06:40.340890 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 19:06:40.340904 9844 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.051s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.082s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.05s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.01s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.159s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.119s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0406 19:06:46.428867 11353 files.go:57] Writing file "/tmp/fitest197612333/SubDir/test1.tmp" I0406 19:06:46.429057 11353 files.go:113] Changing file mode for "/tmp/fitest197612333/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.105s === RUN TestBootstrapChannelBuilder_BuildTasks I0406 19:06:52.694822 12245 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:52.694930 12245 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:52.694962 12245 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:52.695005 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:52.695034 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:52.695047 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:52.695064 12245 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:52.695086 12245 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:52.695102 12245 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:52.701128 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:52.761298 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:52.847146 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:52.913069 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:52.973378 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:53.034044 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:53.090207 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:53.144989 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:53.227351 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:53.295715 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:54.395400 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.621806 12245 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (3.01s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0406 19:06:55.700538 12245 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.700619 12245 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.700668 12245 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.700720 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.700761 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.700789 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.700817 12245 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.700850 12245 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.700877 12245 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.700967 12245 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0406 19:06:55.702817 12245 zones.go:93] ListHostedZonesPages { } I0406 19:06:55.774624 12245 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.07s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0406 19:06:55.774994 12245 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:55.775081 12245 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 19:06:55.775112 12245 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 19:06:55.775144 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 19:06:55.775170 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 19:06:55.775187 12245 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 19:06:55.775204 12245 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0406 19:06:55.775226 12245 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 19:06:55.775241 12245 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 19:06:55.776414 12245 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.14s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.06s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0406 19:06:56.011259 12245 new_cluster.go:935] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Kubenet --- PASS: TestPopulateCluster_Kubenet (0.00s) === RUN TestPopulateCluster_CNI --- PASS: TestPopulateCluster_CNI (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0406 19:06:56.064154 12245 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0406 19:06:56.064198 12245 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0406 19:06:56.064217 12245 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0406 19:06:56.064248 12245 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0406 19:06:56.064265 12245 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet a I0406 19:06:56.064296 12245 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0406 19:06:56.064341 12245 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet b I0406 19:06:56.064370 12245 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0406 19:06:56.064388 12245 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0406 19:06:56.064418 12245 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0406 19:06:56.064434 12245 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0406 19:06:56.064450 12245 subnets.go:182] Assigned CIDR 10.96.0.0/11 to subnet b I0406 19:06:56.064465 12245 subnets.go:182] Assigned CIDR 10.128.0.0/11 to subnet c I0406 19:06:56.064480 12245 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet c I0406 19:06:56.064498 12245 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0406 19:06:56.064515 12245 subnets.go:196] Assigned CIDR 10.4.0.0/14 to subnet b I0406 19:06:56.064531 12245 subnets.go:196] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 3.449s ? k8s.io/kops/upup/pkg/fi/cloudup/alitasks [no test files] === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/aliup 0.087s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0406 19:06:50.607555 12073 target.go:112] Terraform output is in /tmp/kops-render-2474828855 I0406 19:06:50.608820 12073 target.go:112] Terraform output is in /tmp/kops-render-2474828855 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I0406 19:06:50.609568 12073 target.go:168] Cloudformation output is in /tmp/kops-render-3960629859 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0406 19:06:50.610132 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0406 19:06:50.610198 12073 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0406 19:06:50.610257 12073 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0406 19:06:50.610404 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.610481 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.610558 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.610572 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.610697 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.610737 12073 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 19:06:50.610770 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.610847 12073 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 19:06:50.610923 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 19:06:50.611031 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.611079 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.611112 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.611125 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.611152 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.611195 12073 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 19:06:50.611247 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0406 19:06:50.611519 12073 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 19:06:50.611562 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.611633 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 19:06:50.611661 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.611678 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.611700 12073 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0406 19:06:50.611809 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.611936 12073 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 19:06:50.611981 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.611999 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.612022 12073 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0406 19:06:50.612069 12073 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.612177 12073 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0406 19:06:50.612219 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.612237 12073 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0406 19:06:50.612268 12073 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0406 19:06:50.612370 12073 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 19:06:50.612409 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.612443 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.612455 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.612473 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.612493 12073 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0406 19:06:50.612528 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.612571 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.612592 12073 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0406 19:06:50.612627 12073 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.612663 12073 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0406 19:06:50.612682 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 19:06:50.612713 12073 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0406 19:06:50.612829 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0406 19:06:50.612856 12073 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0406 19:06:50.612872 12073 internetgateways.go:66] CreateInternetGateway: { } I0406 19:06:50.612885 12073 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0406 19:06:50.612970 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.613007 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.613037 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.613049 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.613096 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.613115 12073 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 19:06:50.613143 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.613200 12073 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 19:06:50.613265 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 19:06:50.613357 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.613393 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.613424 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.613436 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.613462 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.613497 12073 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 19:06:50.613542 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0406 19:06:50.614146 12073 target.go:168] Cloudformation output is in /tmp/kops-render-1759533107 I0406 19:06:50.614338 12073 target.go:168] Cloudformation output is in /tmp/kops-render-1759533107 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0406 19:06:50.616298 12073 target.go:112] Terraform output is in /tmp/kops-render-227470335 I0406 19:06:50.617392 12073 target.go:112] Terraform output is in /tmp/kops-render-227470335 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0406 19:06:50.617853 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.617906 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.617994 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 19:06:50.618033 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.618053 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.618095 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.618175 12073 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0406 19:06:50.618348 12073 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0406 19:06:50.618395 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 19:06:50.618422 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 19:06:50.618520 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.618558 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.618595 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.618608 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.618626 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.618646 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.618682 12073 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0406 19:06:50.618736 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0406 19:06:50.619025 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.619065 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.619128 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 19:06:50.619155 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.619172 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.619191 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.619229 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.619298 12073 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 19:06:50.619326 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.619343 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.619363 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 19:06:50.619462 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.619501 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.619534 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.619546 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.619578 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.619601 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.619637 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.619677 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.619697 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0406 19:06:50.619892 12073 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 19:06:50.619930 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.619992 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 19:06:50.620020 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.620036 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.620057 12073 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 19:06:50.620109 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.620231 12073 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 19:06:50.620290 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.620320 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.620339 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.620447 12073 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:06:50.620481 12073 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0406 19:06:50.620587 12073 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 19:06:50.620624 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.620657 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.620670 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.620688 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.620710 12073 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 19:06:50.620754 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.620808 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.620825 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.620862 12073 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0406 19:06:50.621065 12073 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 19:06:50.621102 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.621163 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 19:06:50.621190 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.621206 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.621226 12073 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 19:06:50.621278 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.621416 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.621555 12073 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 19:06:50.621471 12073 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 19:06:50.621602 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.621624 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.621642 12073 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0406 19:06:50.621741 12073 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 19:06:50.621774 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.621806 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.621818 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.621836 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.621854 12073 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 19:06:50.621894 12073 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 19:06:50.621952 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.622039 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.622070 12073 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0406 19:06:50.622251 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0406 19:06:50.622355 12073 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0406 19:06:50.622525 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.622579 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.622628 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.622648 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.622734 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.622772 12073 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 19:06:50.622817 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.622861 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.622972 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 19:06:50.623003 12073 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 19:06:50.623042 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 19:06:50.623193 12073 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 19:06:50.623243 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.623291 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.623313 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.623353 12073 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 19:06:50.623396 12073 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 19:06:50.623461 12073 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0406 19:06:50.623727 12073 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 19:06:50.623778 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.623883 12073 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 19:06:50.623927 12073 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 19:06:50.623956 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.623984 12073 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 19:06:50.624097 12073 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 19:06:50.624147 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 19:06:50.624200 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.624219 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.624251 12073 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 19:06:50.624277 12073 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0406 19:06:50.624483 12073 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0406 19:06:50.625148 12073 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 19:06:50.626016 12073 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0406 19:06:50.626383 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 19:06:50.626403 12073 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 19:06:50.626447 12073 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.059s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.036s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.038s === RUN TestDiskRenderAzure I0406 19:06:47.133935 11566 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0406 19:06:47.134334 11566 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0406 19:06:47.134582 11566 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0406 19:06:47.135147 11566 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0406 19:06:47.135400 11566 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0406 19:06:47.135694 11566 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0406 19:06:47.135909 11566 resourcegroup.go:103] Creating a new Resource Group with name: rg I0406 19:06:47.135924 11566 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0406 19:06:47.136136 11566 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0406 19:06:47.136674 11566 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0406 19:06:47.136935 11566 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0406 19:06:47.137195 11566 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0406 19:06:47.137531 11566 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0406 19:06:47.137846 11566 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.059s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.104s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gcetasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.058s === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.023s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.133s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.040s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.105s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0406 19:06:51.858362 12143 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0406 19:06:51.858467 12143 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0406 19:06:51.858487 12143 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0406 19:06:51.858507 12143 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0406 19:06:51.858524 12143 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0406 19:06:51.858543 12143 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0406 19:06:51.858562 12143 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0406 19:06:51.859313 12143 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.065s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.033s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.025s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.035s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.018s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.026s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0406 19:06:48.492101 11875 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.020s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.066s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.003s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.003s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.018s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0406 19:06:52.727277 12257 s3context.go:89] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0406 19:06:52.727323 12257 s3context.go:237] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.048s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... kubernetes so:libc.musl-x86_64.so.1 >>> kops*: Package size: 153.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.23.0-r0.apk >>> kops: Build complete at Wed, 06 Apr 2022 19:07:52 +0000 elapsed time 0h 3m 18s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20220406.190436) (2/3) Purging kubernetes (1.23.5-r1) (3/3) Purging go (1.18-r1) Executing busybox-1.35.0-r7.trigger OK: 331 MiB in 99 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...