>>> kops: Building testing/kops 1.23.0-r1 (using abuild 3.9.0-r0) started Thu, 14 Apr 2022 05:10:55 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.23.5-r2) (2/3) Installing go (1.18.1-r1) (3/3) Installing .makedepends-kops (20220414.051056) Executing busybox-1.35.0-r8.trigger OK: 711 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 76 28.7M 76 21.8M 0 0 72.4M 0 --:--:-- --:--:-- --:--:-- 72.4M 100 28.7M 100 28.7M 0 0 79.3M 0 --:--:-- --:--:-- --:--:-- 79.3M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz >>> kops: Checking sha512sums... kops-1.23.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.23.0.tar.gz... go: downloading k8s.io/klog/v2 v2.30.0 go: downloading github.com/aws/aws-sdk-go v1.43.11 go: downloading k8s.io/client-go v0.23.1 go: downloading github.com/prometheus/client_golang v1.11.0 go: downloading k8s.io/component-base v0.23.1 go: downloading google.golang.org/api v0.57.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/apimachinery v0.23.1 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v0.24.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/blang/semver v3.5.1+incompatible go: downloading k8s.io/api v0.23.1 go: downloading github.com/jetstack/cert-manager v1.6.1 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/digitalocean/godo v1.65.0 go: downloading github.com/spf13/cobra v1.2.1 go: downloading golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 go: downloading github.com/spf13/viper v1.8.1 go: downloading cloud.google.com/go v0.97.0 go: downloading k8s.io/cli-runtime v0.23.1 go: downloading github.com/go-logr/logr v1.2.0 go: downloading github.com/google/go-cmp v0.5.6 go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading github.com/google/go-containerregistry v0.7.0 go: downloading k8s.io/legacy-cloud-providers v0.23.1 go: downloading golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd go: downloading sigs.k8s.io/controller-runtime v0.11.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/stretchr/testify v1.7.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.0.2 go: downloading golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3 go: downloading helm.sh/helm/v3 v3.7.1 go: downloading k8s.io/kubectl v0.23.1 go: downloading github.com/pelletier/go-toml v1.9.3 go: downloading k8s.io/kubelet v0.23.1 go: downloading k8s.io/mount-utils v0.23.1 go: downloading k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.28.0 go: downloading github.com/google/go-querystring v1.0.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.13.0 go: downloading github.com/denverdino/aliyungo v0.0.0-20210425065611-55bee4942cba go: downloading github.com/go-ini/ini v1.62.0 go: downloading github.com/hashicorp/vault/api v1.1.1 go: downloading github.com/pkg/sftp v1.13.0 go: downloading github.com/fsnotify/fsnotify v1.5.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/magiconair/properties v1.8.5 go: downloading github.com/mitchellh/mapstructure v1.4.1 go: downloading github.com/spf13/afero v1.6.0 go: downloading github.com/spf13/cast v1.3.1 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/subosito/gotenv v1.2.0 go: downloading gopkg.in/ini.v1 v1.62.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/googleapis/gnostic v0.5.5 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.23.1 go: downloading github.com/docker/cli v20.10.10+incompatible go: downloading github.com/Azure/azure-sdk-for-go v56.2.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.7 go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c go: downloading github.com/sergi/go-diff v1.2.0 go: downloading gopkg.in/square/go-jose.v2 v2.5.1 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading google.golang.org/protobuf v1.27.1 go: downloading github.com/golang/protobuf v1.5.2 go: downloading golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac go: downloading github.com/spotinst/spotinst-sdk-go v1.85.0 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b go: downloading github.com/aliyun/alibaba-cloud-sdk-go v1.61.1059 go: downloading github.com/Azure/go-autorest/autorest v0.11.19 go: downloading google.golang.org/grpc v1.42.0 go: downloading github.com/google/go-tpm-tools v0.3.0-beta1 go: downloading github.com/google/go-tpm v0.3.2 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/hashicorp/hcl/v2 v2.10.1 go: downloading github.com/zclconf/go-cty v1.8.2 go: downloading k8s.io/csi-translation-lib v0.23.1 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.1 go: downloading k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.1 go: downloading github.com/prometheus/procfs v0.6.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/russross/blackfriday v1.5.2 go: downloading sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.14 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.0.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.1 go: downloading github.com/hashicorp/go-multierror v1.1.0 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/vault/sdk v0.2.1 go: downloading github.com/kr/fs v0.1.0 go: downloading golang.org/x/text v0.3.7 go: downloading k8s.io/apiextensions-apiserver v0.23.1 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.1.4 go: downloading github.com/oklog/ulid v1.3.1 go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.1.1 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.10.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.0 go: downloading k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 go: downloading github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/docker/distribution v2.7.1+incompatible go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.17.0 go: downloading go.uber.org/goleak v1.1.12 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/smartystreets/goconvey v1.6.4 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/hashicorp/go-hclog v0.16.1 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.10.0 go: downloading github.com/go-logr/zapr v1.2.0 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/googleapis/gax-go/v2 v2.1.0 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.2 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.1 go: downloading github.com/stretchr/objx v0.2.0 go: downloading github.com/ghodss/yaml v1.0.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/moby/term v0.0.0-20210610120745-9d4ed1856297 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/form3tech-oss/jwt-go v3.2.3+incompatible go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/mitchellh/reflectwalk v1.0.1 go: downloading github.com/armon/go-metrics v0.3.3 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.35 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/opencontainers/image-spec v1.0.2-0.20210730191737-8e42a01fb1b7 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/docker/docker v20.10.10+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/google/go-attestation v0.3.2 go: downloading google.golang.org/genproto v0.0.0-20211111162719-482062a4217b go: downloading github.com/fatih/color v1.12.0 go: downloading github.com/mattn/go-isatty v0.0.12 go: downloading github.com/mattn/go-colorable v0.1.8 go: downloading github.com/containerd/containerd v1.5.7 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading oras.land/oras-go v0.4.0 go: downloading gotest.tools v2.2.0+incompatible go: downloading github.com/distribution/distribution/v3 v3.0.0-20210804104954-38ab4c606ee3 go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/fvbommel/sortorder v1.0.1 go: downloading github.com/klauspost/compress v1.13.6 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/creack/pty v1.1.11 go: downloading go.uber.org/atomic v1.7.0 go: downloading go.uber.org/multierr v1.6.0 go: downloading github.com/golang/snappy v0.0.3 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.1 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/hashicorp/go-immutable-radix v1.1.0 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/google/certificate-transparency-go v1.1.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/jtolds/gls v4.20.0+incompatible go: downloading github.com/smartystreets/assertions v1.1.0 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading rsc.io/letsencrypt v0.0.3 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/Microsoft/hcsshim v0.8.22 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/moby/sys/mountinfo v0.4.1 go: downloading github.com/Microsoft/go-winio v0.5.1 go: downloading golang.org/x/mod v0.5.1 go: downloading github.com/containerd/cgroups v1.0.1 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/frankban/quicktest v1.11.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 go: downloading github.com/Masterminds/sprig v2.22.0+incompatible go: downloading github.com/cpuguy83/go-md2man v1.0.10 go: downloading github.com/Masterminds/semver v1.5.0 go: downloading github.com/cespare/xxhash v1.1.0 go: downloading github.com/apparentlymart/go-textseg v1.0.0 /bin/sh: tools/get_version.sh: not found go build -ldflags=all="-X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220328-1292-g9a88fe79c0 " -o /home/buildozer/aports/testing/kops/src/kops-1.23.0/.build/local/kops k8s.io/kops/cmd/kops/ go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0414 05:12:54.930245 26315 addon.go:218] addon test wants to update all nodes I0414 05:12:54.932187 26315 addon.go:218] addon test wants to update worker nodes I0414 05:12:54.932542 26315 addon.go:218] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0414 05:12:54.932991 26315 addon.go:251] installing PKI for "test" I0414 05:12:55.177335 26315 addon.go:251] installing PKI for "test" --- PASS: Test_InstallPKI (1.21s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.355s ? k8s.io/kops/channels/pkg/cmd [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0414 05:13:13.401131 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.401210 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.401248 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.401298 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.401347 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.401363 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.401399 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.401426 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.401443 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.401489 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.401527 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.497977 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.498136 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.498831 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:13.521106 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.521153 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.521198 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.521240 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.521276 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.521305 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.521335 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.521366 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.521390 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.521443 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.521478 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.632125 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.632247 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.632637 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:13.640438 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.640484 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.640520 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.640561 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.640596 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.640618 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.640668 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.640705 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.640732 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.640823 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.640867 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.673223 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.673276 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.673456 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:13.679225 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.679255 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.679276 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.679301 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.679324 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.679342 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.679360 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.679376 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.679390 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.679420 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.679441 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.709300 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.709351 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.709535 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:13.716761 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.716825 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.716870 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.716914 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.716954 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.716980 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.717008 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.717039 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.717062 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.717142 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.717181 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.811681 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.811755 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.812024 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:13.817645 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.817675 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.817700 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.817726 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.817750 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.817768 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.817786 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.817806 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.817827 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.817878 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.817903 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.846302 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.846375 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.846621 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.45s) === RUN TestCreateClusterOverride I0414 05:13:13.852511 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.852541 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.852567 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.852595 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.852620 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.852639 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.852656 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.852677 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.852692 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.852749 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.852774 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.910663 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.911145 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.911444 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.07s) === RUN TestCreateClusterComplex I0414 05:13:13.919327 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.919365 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.919399 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.919435 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.919465 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.919489 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.919518 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.919542 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.919563 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.919625 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.919660 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.969522 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:13.969583 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:13.969786 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.06s) === RUN TestCreateClusterHA I0414 05:13:13.975431 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:13.975457 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:13.975479 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:13.975505 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:13.975528 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:13.975545 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:13.975562 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:13.975579 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:13.975593 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:13.975635 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:13.975658 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.009769 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.009821 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.009833 29933 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0414 05:13:14.009842 29933 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0414 05:13:14.010051 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:14.020419 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.020461 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.020489 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.020518 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.020543 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.020561 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.020580 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.020599 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.020618 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.020683 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.020707 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.200269 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.200358 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.200372 29933 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0414 05:13:14.200382 29933 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0414 05:13:14.200681 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.24s) === RUN TestCreateClusterHAGCE I0414 05:13:14.215931 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.215974 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.216014 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.216059 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.216096 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.216126 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.216158 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.216186 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.216211 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.216279 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.216315 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0414 05:13:14.319396 29933 new_cluster.go:372] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0414 05:13:14.319419 29933 new_cluster.go:373] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0414 05:13:14.319490 29933 new_cluster.go:1102] Cloud Provider ID = gce I0414 05:13:14.319575 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.11s) === RUN TestCreateClusterGCE I0414 05:13:14.328638 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.328670 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.328693 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.328720 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.328744 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.328764 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.328785 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.328804 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.328820 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.328868 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.328893 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.378529 29933 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0414 05:13:14.378613 29933 new_cluster.go:1102] Cloud Provider ID = gce I0414 05:13:14.378672 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.06s) === RUN TestCreateClusterHASharedZone I0414 05:13:14.385932 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.385975 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.386009 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.386048 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.386081 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.386106 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.386130 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.386156 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.386179 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.386241 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.386270 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0414 05:13:14.417994 29933 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0414 05:13:14.418035 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.418098 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.418329 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.04s) === RUN TestCreateClusterHASharedZones I0414 05:13:14.425574 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.425602 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.425624 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.425649 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.425671 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.425688 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.425707 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.425724 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.425739 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.425782 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.425805 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0414 05:13:14.460827 29933 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0414 05:13:14.460872 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.460916 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.460930 29933 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0414 05:13:14.461148 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.05s) === RUN TestCreateClusterPrivate I0414 05:13:14.473010 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.473044 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.473071 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.473102 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.473126 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.473145 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.473164 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.473187 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.473203 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.473257 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.473281 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.513001 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.513088 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.513101 29933 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0414 05:13:14.513388 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:14.513418 29933 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.05s) === RUN TestCreateClusterPrivateGCE I0414 05:13:14.520304 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.520334 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.520358 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.520386 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.520410 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.520435 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.520454 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.520475 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.520491 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.520536 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.520558 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.549568 29933 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0414 05:13:14.549627 29933 new_cluster.go:1102] Cloud Provider ID = gce I0414 05:13:14.549672 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.04s) === RUN TestCreateClusterWithNGWSpecified I0414 05:13:14.557618 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.557734 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.558201 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.558257 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.558303 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.558332 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.558366 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.558398 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.558426 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.558505 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.558548 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.671460 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.671516 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.671529 29933 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0414 05:13:14.671752 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:14.671780 29933 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.12s) === RUN TestCreateClusterWithINGWSpecified I0414 05:13:14.678057 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.678082 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.678104 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.678129 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.678151 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.678169 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.678188 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.678204 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.678218 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.678262 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.678285 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.738368 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.738481 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:14.738498 29933 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0414 05:13:14.738868 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:14.738909 29933 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.07s) === RUN TestCreateClusterSharedVPC I0414 05:13:14.748559 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.748609 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.748640 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.748669 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.748694 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.748711 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.748728 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.748748 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.748764 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.748811 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.748836 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.825176 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.825231 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.825269 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.825305 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.825322 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.825362 29933 subnets.go:182] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0414 05:13:14.825597 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.08s) === RUN TestCreateClusterSharedSubnets I0414 05:13:14.830889 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.830916 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.830939 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.830963 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.830985 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.831002 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.831020 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.831035 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.831050 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.831090 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.831112 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.862117 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.862144 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.862193 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.862224 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.862244 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.862267 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.862281 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.862471 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.04s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0414 05:13:14.867674 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.867698 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.867720 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.867746 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.867768 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.867784 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.867802 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.867820 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.867834 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.867880 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.867904 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.893147 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:14.893187 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.893206 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.893262 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.893296 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.893315 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.893339 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.893354 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.893571 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.03s) === RUN TestCreateClusterPrivateSharedSubnets I0414 05:13:14.898751 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.898775 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.898796 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.898823 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.898844 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.898861 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.898879 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.898894 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.898908 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.898950 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.898972 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.979431 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.979459 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.979512 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.979529 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.979556 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:14.979586 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.979604 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.979626 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:14.979640 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:14.979853 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.09s) === RUN TestCreateClusterIPv6 I0414 05:13:14.985189 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:14.985214 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:14.985235 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:14.985257 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:14.985276 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:14.985294 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:14.985310 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:14.985327 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:14.985340 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:14.985378 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:14.985400 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0414 05:13:15.011993 29933 new_cluster.go:1102] Cloud Provider ID = aws I0414 05:13:15.012044 29933 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0414 05:13:15.012287 29933 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.03s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0414 05:13:15.020281 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:15.020328 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:15.020370 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:15.020418 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:15.020511 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:15.020547 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:15.020578 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:15.020612 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:15.020654 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:15.042202 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:15.045528 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:15.045677 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0414 05:13:17.431326 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:19.946672 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:19.946934 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:19.948592 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:19.950719 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:19.951295 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:19.951313 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:19.952415 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:19.952480 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:19.955820 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:19.956640 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:19.956850 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:19.957501 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:19.958047 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:19.958118 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:19.958475 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:19.988999 29933 target.go:112] Terraform output is in /tmp/test3912850075/out I0414 05:13:19.991426 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:19.991470 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:19.991503 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:19.991539 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:19.991568 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:19.991585 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:19.991603 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:19.991625 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:19.991645 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:20.006911 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:20.008556 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:20.008621 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0414 05:13:20.048853 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:20.115600 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0414 05:13:20.115875 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:20.116070 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:20.116152 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:20.116173 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:20.116250 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:20.116245 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:20.116357 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:20.116371 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:20.116426 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:20.119825 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:20.122755 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:20.123051 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:20.123234 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:20.123874 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:20.124433 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:20.124510 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:20.124655 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:20.127171 29933 target.go:168] Cloudformation output is in /tmp/test2789701156/out --- PASS: TestMinimal (5.11s) === RUN TestMinimal_v1_23 I0414 05:13:20.134107 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:20.134186 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:20.134358 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:20.134453 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:20.134550 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:20.134619 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:20.135217 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:20.135263 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:20.135282 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:20.149856 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* I0414 05:13:20.151571 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:20.151630 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.341358 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:21.452979 29933 executor.go:111] Tasks: 0 done / 91 total; 46 can run W0414 05:13:21.453255 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:21.453375 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:21.453512 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:21.453528 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:21.453600 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:21.454252 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:21.457499 29933 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0414 05:13:21.461866 29933 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0414 05:13:21.462288 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.462490 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.463780 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.464511 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.464595 29933 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0414 05:13:21.464925 29933 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0414 05:13:21.524206 29933 target.go:112] Terraform output is in /tmp/test3139053970/out I0414 05:13:21.528677 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:21.528744 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:21.528798 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:21.528856 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:21.528904 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:21.528934 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:21.528968 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:21.529003 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:21.529033 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:21.548525 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:21.550510 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.550585 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0414 05:13:21.590535 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:21.684311 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0414 05:13:21.684689 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:21.684839 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:21.684956 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0414 05:13:21.684969 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:21.684975 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:21.685371 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:21.684985 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:21.685543 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:21.685441 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:21.689953 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:21.692847 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:21.693111 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.693348 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.694268 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.694685 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.694761 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:21.695057 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:21.698374 29933 target.go:168] Cloudformation output is in /tmp/test578253937/out --- PASS: TestMinimal_v1_23 (1.57s) === RUN TestNvidia I0414 05:13:21.708474 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:21.708531 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:21.708574 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:21.708626 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:21.708708 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:21.708737 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:21.708771 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:21.708808 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:21.708838 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:21.726299 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:21.727843 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.727894 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.764592 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:21.862073 29933 executor.go:111] Tasks: 0 done / 87 total; 45 can run W0414 05:13:21.862227 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:21.862686 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:21.862707 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:21.862720 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:21.862714 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:21.862815 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:21.864672 29933 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0414 05:13:21.870554 29933 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0414 05:13:21.870771 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.870933 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.871483 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.872230 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.872291 29933 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0414 05:13:21.872462 29933 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0414 05:13:21.914826 29933 target.go:112] Terraform output is in /tmp/test3867036309/out I0414 05:13:21.919185 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:21.919248 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:21.919300 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:21.919354 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:21.919401 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:21.919431 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:21.919462 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:21.919499 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:21.919528 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:21.944074 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:21.946979 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.947096 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:21.983758 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:22.051157 29933 executor.go:111] Tasks: 0 done / 87 total; 45 can run I0414 05:13:22.051457 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:22.051473 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:22.051526 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:22.051633 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:22.051767 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:22.051782 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:22.051879 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0414 05:13:22.051665 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:22.052072 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:22.053718 29933 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0414 05:13:22.061905 29933 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0414 05:13:22.062129 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.062299 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.062827 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.064839 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.064977 29933 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0414 05:13:22.065261 29933 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0414 05:13:22.068141 29933 target.go:168] Cloudformation output is in /tmp/test2567469741/out --- PASS: TestNvidia (0.37s) === RUN TestMinimalGossip I0414 05:13:22.079340 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:22.079389 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:22.079434 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:22.079479 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:22.079525 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:22.079554 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:22.079583 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:22.079619 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:22.079647 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:22.107648 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.107745 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.145783 29933 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0414 05:13:22.254486 29933 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0414 05:13:22.254862 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:22.255053 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:22.255399 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:22.257881 29933 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0414 05:13:22.262886 29933 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0414 05:13:22.263177 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.263561 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.266803 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.269229 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.269374 29933 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0414 05:13:22.269797 29933 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0414 05:13:22.318834 29933 target.go:112] Terraform output is in /tmp/test1429929812/out --- PASS: TestMinimalGossip (0.24s) === RUN TestMinimalGossipIRSA I0414 05:13:22.323031 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:22.323093 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:22.323141 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:22.323198 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:22.323243 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:22.323272 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:22.323303 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:22.323339 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:22.323369 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:22.350400 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.350497 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.388122 29933 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0414 05:13:22.460100 29933 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0414 05:13:22.460422 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:22.461535 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:22.462416 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:22.466114 29933 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0414 05:13:22.470987 29933 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0414 05:13:22.471211 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.471387 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.471935 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.474326 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:22.474448 29933 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0414 05:13:22.474768 29933 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0414 05:13:22.518372 29933 target.go:112] Terraform output is in /tmp/test1214915951/out --- PASS: TestMinimalGossipIRSA (0.20s) === RUN TestMinimalGCE I0414 05:13:22.520776 29933 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0414 05:13:22.521841 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0414 05:13:23.356817 29933 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0414 05:13:23.356949 29933 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0414 05:13:23.357110 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0414 05:13:23.357180 29933 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0414 05:13:23.357224 29933 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0414 05:13:23.357262 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:23.357388 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:23.357436 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:23.359087 29933 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0414 05:13:23.361549 29933 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0414 05:13:23.365956 29933 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0414 05:13:23.366819 29933 instancetemplate.go:555] adding csa: default I0414 05:13:23.367539 29933 instancetemplate.go:555] adding csa: default I0414 05:13:23.367609 29933 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0414 05:13:23.367820 29933 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0414 05:13:23.384970 29933 target.go:112] Terraform output is in /tmp/test3570190934/out --- PASS: TestMinimalGCE (0.87s) === RUN TestMinimalGCEPrivate I0414 05:13:23.387315 29933 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0414 05:13:23.388425 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0414 05:13:24.133839 29933 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0414 05:13:24.133890 29933 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0414 05:13:24.133990 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0414 05:13:24.134261 29933 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0414 05:13:24.134622 29933 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0414 05:13:24.134669 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:24.134775 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:24.134791 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:24.137282 29933 executor.go:111] Tasks: 0 done / 63 total; 37 can run I0414 05:13:24.140496 29933 executor.go:111] Tasks: 37 done / 63 total; 18 can run I0414 05:13:24.145721 29933 executor.go:111] Tasks: 55 done / 63 total; 4 can run I0414 05:13:24.148396 29933 instancetemplate.go:555] adding csa: default I0414 05:13:24.148512 29933 executor.go:111] Tasks: 59 done / 63 total; 3 can run I0414 05:13:24.149791 29933 instancetemplate.go:555] adding csa: default I0414 05:13:24.149902 29933 executor.go:111] Tasks: 62 done / 63 total; 1 can run I0414 05:13:24.150040 29933 executor.go:111] Tasks: 63 done / 63 total; 0 can run I0414 05:13:24.175571 29933 target.go:112] Terraform output is in /tmp/test641308467/out --- PASS: TestMinimalGCEPrivate (0.79s) === RUN TestHA I0414 05:13:24.178474 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:24.178509 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:24.178538 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:24.178567 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:24.178592 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:24.178610 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:24.178627 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:24.178648 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:24.178663 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:24.209877 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:24.212907 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.213010 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.213075 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.213135 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.250480 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:24.337796 29933 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0414 05:13:24.338259 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:24.338343 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:24.338409 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:24.338430 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:24.338484 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:24.338540 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:24.338686 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:24.338887 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:24.344176 29933 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0414 05:13:24.349605 29933 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0414 05:13:24.349841 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.349977 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.350102 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.350274 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.350601 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.352619 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.355745 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.355948 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.356088 29933 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0414 05:13:24.356463 29933 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0414 05:13:24.427233 29933 target.go:112] Terraform output is in /tmp/test1087693572/out --- PASS: TestHA (0.25s) === RUN TestHighAvailabilityGCE I0414 05:13:24.432397 29933 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0414 05:13:24.433563 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0414 05:13:24.581339 29933 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0414 05:13:24.581437 29933 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0414 05:13:24.581572 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0414 05:13:24.581614 29933 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0414 05:13:24.581637 29933 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0414 05:13:24.581657 29933 autoscalinggroup.go:168] gsa: default W0414 05:13:24.581720 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0414 05:13:24.581752 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:24.581771 29933 autoscalinggroup.go:168] gsa: default W0414 05:13:24.581824 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0414 05:13:24.581857 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:24.581881 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:24.581941 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:24.581964 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:24.583949 29933 executor.go:111] Tasks: 0 done / 76 total; 41 can run I0414 05:13:24.588941 29933 executor.go:111] Tasks: 41 done / 76 total; 20 can run I0414 05:13:24.593963 29933 executor.go:111] Tasks: 61 done / 76 total; 7 can run I0414 05:13:24.595286 29933 instancetemplate.go:555] adding csa: default I0414 05:13:24.595319 29933 instancetemplate.go:555] adding csa: default I0414 05:13:24.595414 29933 instancetemplate.go:555] adding csa: default I0414 05:13:24.595485 29933 executor.go:111] Tasks: 68 done / 76 total; 5 can run I0414 05:13:24.596751 29933 instancetemplate.go:555] adding csa: default I0414 05:13:24.597025 29933 executor.go:111] Tasks: 73 done / 76 total; 3 can run I0414 05:13:24.597410 29933 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0414 05:13:24.632677 29933 target.go:112] Terraform output is in /tmp/test1986828386/out --- PASS: TestHighAvailabilityGCE (0.20s) === RUN TestComplex I0414 05:13:24.635481 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:24.635518 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:24.635548 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:24.635583 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:24.635641 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:24.635662 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:24.635688 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:24.635708 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:24.635730 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:24.667067 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:24.670872 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.670988 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.708865 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:24.793141 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0414 05:13:24.793698 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:24.793849 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:24.794039 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:24.794081 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:24.794217 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:24.798848 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:24.803265 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:24.803524 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.803987 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.804270 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.806372 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.806499 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:24.806979 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0414 05:13:24.881650 29933 target.go:112] Terraform output is in /tmp/test3567232801/out I0414 05:13:24.885812 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:24.885897 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:24.885981 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:24.886075 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:24.886145 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:24.886202 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:24.886263 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:24.886329 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:24.886377 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:24.920103 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:24.924386 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.924627 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:24.964709 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:25.066015 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0414 05:13:25.066583 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:25.066606 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:25.066698 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:25.066761 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:25.066777 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:25.069334 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:25.077618 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:25.078073 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.078288 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.079149 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.079659 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.079743 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:25.080048 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0414 05:13:25.084637 29933 target.go:168] Cloudformation output is in /tmp/test3330287981/out I0414 05:13:25.099022 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:25.099079 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:25.099125 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:25.099180 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:25.099225 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:25.099257 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:25.099292 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:25.099327 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:25.099353 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:25.136020 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:25.139766 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.139863 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.175982 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:25.280082 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0414 05:13:25.281296 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:25.281372 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:25.281345 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:25.281488 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:25.281566 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:25.285631 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:25.290908 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:25.291512 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.291792 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.293900 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.295217 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.295360 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:25.295755 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0414 05:13:25.368205 29933 target.go:112] Terraform output is in /tmp/test1438383516/out --- PASS: TestComplex (0.73s) === RUN TestCompress I0414 05:13:25.370478 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:25.370511 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:25.370542 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:25.370572 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:25.370598 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:25.370615 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:25.370645 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:25.370666 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:25.370683 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:25.383941 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:25.385542 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.385597 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.422874 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:25.523039 29933 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0414 05:13:25.523603 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:25.523702 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:25.523883 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:25.523913 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:25.524018 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:25.529533 29933 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0414 05:13:25.534703 29933 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0414 05:13:25.535101 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.535325 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.539450 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.541115 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.542214 29933 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0414 05:13:25.542590 29933 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0414 05:13:25.586496 29933 target.go:112] Terraform output is in /tmp/test3782510856/out --- PASS: TestCompress (0.22s) === RUN TestExternalPolicies I0414 05:13:25.590715 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:25.590768 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:25.590815 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:25.590862 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:25.590901 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:25.590927 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:25.590953 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:25.590983 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:25.591016 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:25.620391 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:25.623437 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.623530 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.661776 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:25.767986 29933 executor.go:111] Tasks: 0 done / 99 total; 46 can run W0414 05:13:25.768250 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:25.768561 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:25.768688 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:25.768789 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:25.768809 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:25.768905 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:25.773847 29933 executor.go:111] Tasks: 46 done / 99 total; 20 can run I0414 05:13:25.778096 29933 executor.go:111] Tasks: 66 done / 99 total; 30 can run I0414 05:13:25.778498 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.778681 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.779811 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.779914 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.780007 29933 executor.go:111] Tasks: 96 done / 99 total; 3 can run I0414 05:13:25.780256 29933 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0414 05:13:25.820835 29933 target.go:112] Terraform output is in /tmp/test3782871300/out --- PASS: TestExternalPolicies (0.23s) === RUN TestMinimalIPv6 I0414 05:13:25.823264 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:25.823298 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:25.823328 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:25.823358 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:25.823411 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:25.823428 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:25.823446 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:25.823467 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:25.823484 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:25.837642 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:25.839423 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.839476 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:25.876649 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:26.028198 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0414 05:13:26.028540 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:26.028824 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:26.028852 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:26.028951 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:26.029043 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:26.029069 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:26.030723 29933 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0414 05:13:26.034136 29933 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0414 05:13:26.035635 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.036055 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.038548 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.039184 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.039273 29933 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0414 05:13:26.039709 29933 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0414 05:13:26.039939 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:13:26.079059 29933 target.go:112] Terraform output is in /tmp/test227908527/out I0414 05:13:26.083451 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:26.083513 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:26.083567 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:26.083625 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:26.083672 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:26.083701 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:26.083736 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:26.083771 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:26.083801 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:26.108663 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:26.111922 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.112018 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.150556 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:26.296600 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0414 05:13:26.297201 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:26.297384 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0414 05:13:26.297399 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:26.297406 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:26.297506 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:26.297578 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:26.297530 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:26.297617 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:26.297739 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:26.299345 29933 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0414 05:13:26.302846 29933 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0414 05:13:26.303089 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.303453 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.305502 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.308565 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.308685 29933 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0414 05:13:26.309115 29933 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0414 05:13:26.309380 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:13:26.313160 29933 target.go:168] Cloudformation output is in /tmp/test3302975890/out --- PASS: TestMinimalIPv6 (0.50s) === RUN TestMinimalIPv6Private I0414 05:13:26.324655 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:26.324708 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:26.324753 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:26.324805 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:26.324884 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:26.324913 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:26.324943 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:26.324975 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:26.325002 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:26.350863 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:26.353561 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.353638 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.391424 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:26.566015 29933 executor.go:111] Tasks: 0 done / 118 total; 46 can run I0414 05:13:26.567033 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:26.567169 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:26.568358 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:26.568914 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:26.568940 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:26.569042 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:26.571547 29933 executor.go:111] Tasks: 46 done / 118 total; 23 can run I0414 05:13:26.577124 29933 executor.go:111] Tasks: 69 done / 118 total; 34 can run I0414 05:13:26.577706 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.577878 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.579995 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.583548 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.583687 29933 executor.go:111] Tasks: 103 done / 118 total; 9 can run I0414 05:13:26.584391 29933 executor.go:111] Tasks: 112 done / 118 total; 6 can run I0414 05:13:26.585000 29933 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0414 05:13:26.643709 29933 target.go:112] Terraform output is in /tmp/test662819745/out --- PASS: TestMinimalIPv6Private (0.32s) === RUN TestMinimalIPv6Calico I0414 05:13:26.646235 29933 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0414 05:13:26.646378 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:26.646412 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:26.646442 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:26.646471 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:26.646524 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:26.646541 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:26.646558 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:26.646578 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:26.646595 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:26.660721 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:26.662557 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.662614 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:26.700176 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:27.035802 29933 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0414 05:13:27.036418 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:27.036582 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:27.036669 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:27.036621 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:27.036744 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:27.036663 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:27.038224 29933 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0414 05:13:27.041581 29933 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0414 05:13:27.041993 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.042171 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.043859 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.044048 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.044287 29933 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0414 05:13:27.044794 29933 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0414 05:13:27.045425 29933 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0414 05:13:27.109521 29933 target.go:112] Terraform output is in /tmp/test3888528375/out I0414 05:13:27.112676 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:27.112710 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:27.112737 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:27.112767 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:27.112791 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:27.112808 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:27.112828 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:27.112848 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:27.112865 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:27.126888 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:27.128707 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.128761 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.166736 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:27.543875 29933 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0414 05:13:27.544112 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:27.544236 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:27.545229 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:27.545300 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:27.545331 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:27.545459 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:27.546070 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:27.546119 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:27.546259 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:27.550374 29933 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0414 05:13:27.555402 29933 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0414 05:13:27.555994 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.556280 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.557147 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.559033 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.559212 29933 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0414 05:13:27.561361 29933 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0414 05:13:27.561665 29933 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0414 05:13:27.566718 29933 target.go:168] Cloudformation output is in /tmp/test715563359/out I0414 05:13:27.578620 29933 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Calico (0.93s) === RUN TestMinimalIPv6Cilium I0414 05:13:27.578741 29933 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0414 05:13:27.578979 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:27.579045 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:27.579093 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:27.579146 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:27.579228 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:27.579257 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:27.579288 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:27.579324 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:27.579353 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:27.604081 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:27.607233 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.607364 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.645492 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:27.795498 29933 executor.go:111] Tasks: 0 done / 106 total; 47 can run W0414 05:13:27.795934 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:27.796056 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:27.796403 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:27.796452 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:27.796437 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:27.796523 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:27.798870 29933 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0414 05:13:27.805314 29933 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0414 05:13:27.805647 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.805732 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.806344 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.807064 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.807140 29933 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0414 05:13:27.807507 29933 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0414 05:13:27.807797 29933 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0414 05:13:27.869035 29933 target.go:112] Terraform output is in /tmp/test3669356563/out I0414 05:13:27.871599 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:27.871636 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:27.871663 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:27.871694 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:27.871721 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:27.871738 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:27.871756 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:27.871776 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:27.871794 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:27.892793 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:27.896693 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.896797 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:27.933435 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:28.121515 29933 executor.go:111] Tasks: 0 done / 106 total; 47 can run I0414 05:13:28.122047 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:28.122161 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:28.122261 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:28.122280 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:28.122307 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:28.122286 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:28.122389 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:28.122433 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } W0414 05:13:28.122577 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:28.124766 29933 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0414 05:13:28.129953 29933 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0414 05:13:28.130383 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.130592 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.133504 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.135140 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.135259 29933 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0414 05:13:28.135661 29933 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0414 05:13:28.135924 29933 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0414 05:13:28.140021 29933 target.go:168] Cloudformation output is in /tmp/test4167240570/out I0414 05:13:28.152953 29933 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Cilium (0.57s) === RUN TestMinimalWarmPool I0414 05:13:28.153274 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:28.153325 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:28.153371 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:28.153421 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:28.153504 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:28.153534 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:28.153565 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:28.153600 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:28.153626 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:28.177997 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:28.181008 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.181084 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.218677 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:28.394653 29933 executor.go:111] Tasks: 0 done / 89 total; 46 can run W0414 05:13:28.395301 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:28.395407 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:28.395457 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:28.395571 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:28.395607 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:28.395727 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:28.397659 29933 executor.go:111] Tasks: 46 done / 89 total; 19 can run I0414 05:13:28.405638 29933 executor.go:111] Tasks: 65 done / 89 total; 21 can run I0414 05:13:28.405831 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.406046 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.406518 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.408597 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.408720 29933 executor.go:111] Tasks: 86 done / 89 total; 2 can run I0414 05:13:28.408987 29933 executor.go:111] Tasks: 88 done / 89 total; 1 can run I0414 05:13:28.409346 29933 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0414 05:13:28.464102 29933 target.go:112] Terraform output is in /tmp/test3274032013/out --- PASS: TestMinimalWarmPool (0.32s) === RUN TestMinimalEtcd I0414 05:13:28.468975 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:28.469035 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:28.469090 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:28.469143 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:28.469236 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:28.469268 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:28.469302 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:28.469341 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:28.469369 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:28.495368 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:28.498465 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.498559 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.535745 29933 zones.go:93] ListHostedZonesPages { } W0414 05:13:28.605989 29933 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0414 05:13:28.606021 29933 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0414 05:13:28.607036 29933 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0414 05:13:28.607121 29933 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0414 05:13:28.607142 29933 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:28.614033 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0414 05:13:28.614459 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:28.614745 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:28.614805 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0414 05:13:28.614840 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0414 05:13:28.614861 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:28.614825 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:28.614873 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:28.614930 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:28.614951 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } I0414 05:13:28.617734 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:28.622548 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:28.622956 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.623176 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.624069 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.625967 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.626135 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:28.626386 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:28.629376 29933 target.go:168] Cloudformation output is in /tmp/test3494785657/out --- PASS: TestMinimalEtcd (0.17s) === RUN TestMinimalGp3 I0414 05:13:28.640424 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:28.640490 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:28.640564 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:28.640662 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:28.640760 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:28.640821 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:28.640882 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:28.640938 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:28.640987 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:28.668578 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:28.671437 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.671536 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.708905 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:28.815452 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:28.815746 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:28.815814 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:28.815780 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:28.815885 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:28.816008 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:28.818224 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:28.822407 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:28.827967 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:28.828344 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.828437 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.829734 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.830029 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.830118 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:28.830451 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:28.866790 29933 target.go:112] Terraform output is in /tmp/test4172697756/out I0414 05:13:28.870923 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:28.870985 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:28.871036 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:28.871093 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:28.871138 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:28.871168 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:28.871199 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:28.871235 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:28.871265 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:28.895913 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:28.898874 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.898971 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:28.936209 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:29.038028 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:29.038491 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:29.038520 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:29.038613 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:29.038729 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:29.038837 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:29.038854 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0414 05:13:29.038865 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:29.038868 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:29.038966 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0414 05:13:29.044437 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:29.049360 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:29.049522 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.049780 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.050191 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.051428 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.051505 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:29.051789 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:29.054890 29933 target.go:168] Cloudformation output is in /tmp/test268366318/out --- PASS: TestMinimalGp3 (0.42s) === RUN TestExistingIAMCloudformation I0414 05:13:29.065208 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:29.065265 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:29.065312 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:29.065364 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:29.065442 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:29.065471 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:29.065505 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:29.065540 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:29.065567 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:29.089662 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:29.091913 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.091993 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.129056 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:29.222344 29933 executor.go:111] Tasks: 0 done / 76 total; 42 can run W0414 05:13:29.222613 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:29.222840 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:29.222869 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:29.222955 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:29.224314 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:29.224845 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:29.224966 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:29.224977 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:29.225041 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:29.229055 29933 executor.go:111] Tasks: 42 done / 76 total; 11 can run I0414 05:13:29.233985 29933 executor.go:111] Tasks: 53 done / 76 total; 21 can run I0414 05:13:29.234373 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.234627 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.236359 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.237884 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.238028 29933 executor.go:111] Tasks: 74 done / 76 total; 2 can run I0414 05:13:29.238542 29933 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0414 05:13:29.240899 29933 target.go:168] Cloudformation output is in /tmp/test371875895/out --- PASS: TestExistingIAMCloudformation (0.18s) === RUN TestExistingSG I0414 05:13:29.249076 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:29.249125 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:29.249171 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:29.249223 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:29.249266 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:29.249296 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:29.249327 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:29.249362 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:29.249391 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:29.284971 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:29.288178 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.288275 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.288341 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.288399 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.325485 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:29.439184 29933 executor.go:111] Tasks: 0 done / 139 total; 50 can run W0414 05:13:29.439464 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:29.440454 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:29.441299 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:29.441319 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:29.441379 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:29.442702 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:29.442864 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:29.442952 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:29.444801 29933 executor.go:111] Tasks: 50 done / 139 total; 26 can run I0414 05:13:29.449026 29933 executor.go:111] Tasks: 76 done / 139 total; 58 can run I0414 05:13:29.449313 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.449439 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.449690 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.450537 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.451115 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.454270 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.454352 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.455495 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.455634 29933 executor.go:111] Tasks: 134 done / 139 total; 5 can run I0414 05:13:29.455994 29933 executor.go:111] Tasks: 139 done / 139 total; 0 can run I0414 05:13:29.536351 29933 target.go:112] Terraform output is in /tmp/test3165225171/out --- PASS: TestExistingSG (0.29s) === RUN TestBastionAdditionalUserData I0414 05:13:29.543338 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:29.543403 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:29.543465 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:29.543522 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:29.543581 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:29.543611 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:29.543645 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:29.543681 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:29.543711 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:29.573793 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:29.576498 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.576580 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.576640 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.613875 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:29.700217 29933 executor.go:111] Tasks: 0 done / 120 total; 49 can run W0414 05:13:29.701816 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:29.701823 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:29.701882 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:29.702057 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:29.702201 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:29.702883 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:29.702899 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:29.706828 29933 executor.go:111] Tasks: 49 done / 120 total; 29 can run I0414 05:13:29.712528 29933 executor.go:111] Tasks: 78 done / 120 total; 36 can run I0414 05:13:29.712836 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.713109 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.713288 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.713523 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.714270 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.714856 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.714929 29933 executor.go:111] Tasks: 114 done / 120 total; 5 can run I0414 05:13:29.715343 29933 executor.go:111] Tasks: 119 done / 120 total; 1 can run I0414 05:13:29.715519 29933 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0414 05:13:29.765960 29933 target.go:112] Terraform output is in /tmp/test211618603/out --- PASS: TestBastionAdditionalUserData (0.23s) === RUN TestPrivateWeave I0414 05:13:29.768932 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:29.768968 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:29.768999 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:29.769029 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:29.769083 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:29.769101 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:29.769118 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:29.769138 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:29.769154 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:29.784600 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:29.786281 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.786336 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.786387 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.823480 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:29.935042 29933 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0414 05:13:29.935874 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:29.936032 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:29.935924 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:29.936080 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:29.936192 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:29.936281 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:29.936138 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:29.943006 29933 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0414 05:13:29.948506 29933 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0414 05:13:29.948776 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.949012 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.949149 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.949371 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.951115 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.952201 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:29.952330 29933 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0414 05:13:29.952825 29933 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0414 05:13:29.952999 29933 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0414 05:13:30.029044 29933 target.go:112] Terraform output is in /tmp/test1176470292/out --- PASS: TestPrivateWeave (0.26s) === RUN TestPrivateFlannel I0414 05:13:30.033434 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:30.033484 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:30.033529 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:30.033575 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:30.033656 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:30.033684 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:30.033711 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:30.033741 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:30.033767 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:30.059189 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:30.061616 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.061672 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.061709 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.099135 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:30.211665 29933 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0414 05:13:30.211939 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:30.212096 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:30.212121 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0414 05:13:30.212175 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:30.212146 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:30.212179 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:30.212571 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:30.214443 29933 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0414 05:13:30.222376 29933 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0414 05:13:30.222600 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.222937 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.223080 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.223302 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.223805 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.224960 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.225191 29933 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0414 05:13:30.226019 29933 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0414 05:13:30.226217 29933 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0414 05:13:30.294174 29933 target.go:112] Terraform output is in /tmp/test1156933558/out --- PASS: TestPrivateFlannel (0.27s) === RUN TestPrivateCalico I0414 05:13:30.298836 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:30.298895 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:30.298946 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:30.299004 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:30.299099 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:30.299128 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:30.299159 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:30.299196 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:30.299226 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:30.326878 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:30.329881 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.329976 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.330037 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.367511 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:30.683682 29933 executor.go:111] Tasks: 0 done / 119 total; 48 can run W0414 05:13:30.684309 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:30.684373 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:30.684523 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:30.684577 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:30.684597 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:30.684575 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:30.684756 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:30.687538 29933 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0414 05:13:30.693800 29933 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0414 05:13:30.694033 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.694161 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.694346 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.694463 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.694791 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.695364 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.695426 29933 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0414 05:13:30.695855 29933 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0414 05:13:30.696027 29933 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0414 05:13:30.762907 29933 target.go:112] Terraform output is in /tmp/test3063414716/out I0414 05:13:30.766229 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:30.766265 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:30.766295 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:30.766327 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:30.766352 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:30.766371 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:30.766389 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:30.766410 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:30.766426 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:30.781921 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:30.783585 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.783638 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.783678 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:30.820992 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:31.126697 29933 executor.go:111] Tasks: 0 done / 119 total; 48 can run W0414 05:13:31.127023 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:31.127220 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:31.127271 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:31.128581 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:31.128608 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:31.128693 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:31.129595 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:31.129823 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:31.129845 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:31.131122 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:31.132931 29933 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0414 05:13:31.135952 29933 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0414 05:13:31.136388 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.136531 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.136831 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.136979 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.137675 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.139494 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.139705 29933 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0414 05:13:31.140043 29933 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0414 05:13:31.140230 29933 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0414 05:13:31.145207 29933 target.go:168] Cloudformation output is in /tmp/test3256422692/out --- PASS: TestPrivateCalico (0.86s) === RUN TestPrivateCilium I0414 05:13:31.156996 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:31.157026 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:31.157051 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:31.157080 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:31.157131 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:31.157149 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:31.157168 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:31.157190 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:31.157206 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:31.172527 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:31.174334 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.174387 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.174424 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.210955 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:31.355718 29933 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0414 05:13:31.357281 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:31.357313 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:31.357703 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:31.357909 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:31.358387 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:31.358497 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:31.358650 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:31.361036 29933 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0414 05:13:31.366447 29933 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0414 05:13:31.366716 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.366897 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.367111 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.367245 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.368004 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.368134 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.368207 29933 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0414 05:13:31.368450 29933 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0414 05:13:31.368540 29933 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0414 05:13:31.433728 29933 target.go:112] Terraform output is in /tmp/test35814079/out I0414 05:13:31.438760 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:31.438829 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:31.438883 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:31.438942 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:31.438989 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:31.439019 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:31.439051 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:31.439089 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:31.439120 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:31.466638 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:31.469833 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.469932 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.469993 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.508034 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:31.631262 29933 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0414 05:13:31.631932 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:31.632008 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:31.632031 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:31.632101 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:31.632177 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:31.632184 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:31.632191 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:31.632350 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:31.632383 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:31.632534 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" },{ Key: "Name", Value: "privatecilium.example.com" }] }] } I0414 05:13:31.634541 29933 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0414 05:13:31.642244 29933 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0414 05:13:31.642572 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.642796 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.642901 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.643071 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.643668 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.644123 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.644203 29933 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0414 05:13:31.644595 29933 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0414 05:13:31.644753 29933 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0414 05:13:31.649168 29933 target.go:168] Cloudformation output is in /tmp/test647276825/out --- PASS: TestPrivateCilium (0.50s) === RUN TestPrivateCilium2 I0414 05:13:31.662132 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:31.662188 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:31.662232 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:31.662284 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:31.662365 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:31.662393 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:31.662423 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:31.662458 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:31.662492 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:31.684602 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0414 05:13:31.686759 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.686813 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:31.686849 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.066877 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:33.208015 29933 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0414 05:13:33.208298 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:33.208505 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:33.208880 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:33.209027 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:33.209043 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:33.209065 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:33.209155 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:33.210578 29933 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0414 05:13:33.213507 29933 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0414 05:13:33.213689 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.213862 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.214108 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.217201 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.217327 29933 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0414 05:13:33.217654 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.220959 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.221104 29933 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0414 05:13:33.221484 29933 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0414 05:13:33.295115 29933 target.go:112] Terraform output is in /tmp/test46398074/out I0414 05:13:33.299875 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:33.299942 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:33.300005 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:33.300078 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:33.300146 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:33.300183 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:33.300229 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:33.300274 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:33.300314 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:33.323895 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0414 05:13:33.325781 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.325841 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.325879 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.364691 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:33.497861 29933 executor.go:111] Tasks: 0 done / 121 total; 49 can run I0414 05:13:33.498140 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:33.498177 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:33.498274 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:33.498290 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:33.498708 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:33.498762 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:33.498817 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:33.498951 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:33.498980 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:33.499133 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0414 05:13:33.500788 29933 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0414 05:13:33.503766 29933 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0414 05:13:33.504232 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.504417 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.504573 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.506415 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.506562 29933 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0414 05:13:33.506706 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.507702 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.507823 29933 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0414 05:13:33.508043 29933 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0414 05:13:33.513666 29933 target.go:168] Cloudformation output is in /tmp/test4158269636/out --- PASS: TestPrivateCilium2 (1.87s) === RUN TestPrivateCiliumAdvanced I0414 05:13:33.528984 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:33.529080 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:33.529194 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:33.529280 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:33.529371 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:33.529401 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:33.529434 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:33.529472 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:33.529501 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:33.561714 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:33.564560 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.564642 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.564701 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.602725 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:33.720325 29933 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0414 05:13:33.720569 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:33.720672 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:33.720872 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:33.721123 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:33.721246 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:33.721281 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:33.721364 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:33.723230 29933 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0414 05:13:33.732105 29933 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0414 05:13:33.732571 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.732736 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.732879 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.732934 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.734090 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.735043 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.735164 29933 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0414 05:13:33.735576 29933 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0414 05:13:33.735738 29933 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0414 05:13:33.804671 29933 target.go:112] Terraform output is in /tmp/test900129626/out I0414 05:13:33.809548 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:33.809617 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:33.809667 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:33.809723 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:33.809771 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:33.809802 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:33.809833 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:33.809873 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:33.809903 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:33.846973 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:33.851193 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.851398 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.851559 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:33.889248 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:34.008826 29933 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0414 05:13:34.009216 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:34.009327 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:34.009462 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:34.009588 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:34.009603 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:34.009681 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:34.009701 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" },{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" }] }] } I0414 05:13:34.009715 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:34.009884 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:34.009890 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:34.011471 29933 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0414 05:13:34.016638 29933 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0414 05:13:34.017984 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.018260 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.018343 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.018388 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.019058 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.019357 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.019440 29933 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0414 05:13:34.019767 29933 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0414 05:13:34.019921 29933 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0414 05:13:34.024317 29933 target.go:168] Cloudformation output is in /tmp/test567766735/out --- PASS: TestPrivateCiliumAdvanced (0.51s) === RUN TestPrivateCanal I0414 05:13:34.038760 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:34.038815 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:34.038860 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:34.038913 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:34.038996 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:34.039024 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:34.039055 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:34.039090 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:34.039120 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:34.068984 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0414 05:13:34.072216 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.072382 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:34.072490 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.169608 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:35.477120 29933 executor.go:111] Tasks: 0 done / 119 total; 49 can run W0414 05:13:35.477528 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:35.477596 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:35.477813 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:35.477898 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:35.477976 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:35.477936 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:35.478092 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:35.479606 29933 executor.go:111] Tasks: 49 done / 119 total; 28 can run I0414 05:13:35.482539 29933 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0414 05:13:35.482901 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.482973 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.483077 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.483146 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.484971 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.487219 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:35.487330 29933 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0414 05:13:35.487618 29933 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0414 05:13:35.487705 29933 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0414 05:13:35.525202 29933 target.go:112] Terraform output is in /tmp/test533737816/out --- PASS: TestPrivateCanal (1.49s) === RUN TestPrivateKopeio I0414 05:13:35.528251 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:35.528287 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:35.528318 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:35.528348 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:35.528400 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:35.528417 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:35.528435 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:35.528454 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:35.528470 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:35.546807 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:35.548521 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.548579 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.548616 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.586739 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:35.686799 29933 executor.go:111] Tasks: 0 done / 124 total; 48 can run W0414 05:13:35.687398 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:35.687496 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:35.687555 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:35.687610 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:35.687643 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:35.687630 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:35.687685 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:35.691115 29933 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0414 05:13:35.698256 29933 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0414 05:13:35.698875 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.699104 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.699475 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.699784 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.700528 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.701546 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.701668 29933 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0414 05:13:35.702362 29933 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0414 05:13:35.767120 29933 target.go:112] Terraform output is in /tmp/test2682417069/out --- PASS: TestPrivateKopeio (0.24s) === RUN TestUnmanaged I0414 05:13:35.769884 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:35.769920 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:35.769950 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:35.769980 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:35.770006 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:35.770023 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:35.770039 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:35.770060 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:35.770076 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:35.792039 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:35.795256 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.795375 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.795474 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.833869 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:35.934923 29933 executor.go:111] Tasks: 0 done / 106 total; 46 can run W0414 05:13:35.935820 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:35.935908 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:35.935978 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:35.936011 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:35.936141 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:35.936160 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:35.936313 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:35.938267 29933 executor.go:111] Tasks: 46 done / 106 total; 25 can run I0414 05:13:35.948197 29933 executor.go:111] Tasks: 71 done / 106 total; 31 can run I0414 05:13:35.948666 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.948794 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.949045 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.949194 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.950311 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.951176 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:35.951318 29933 executor.go:111] Tasks: 102 done / 106 total; 4 can run I0414 05:13:35.951850 29933 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0414 05:13:36.043487 29933 target.go:112] Terraform output is in /tmp/test2165103545/out --- PASS: TestUnmanaged (0.28s) === RUN TestPrivateSharedSubnet I0414 05:13:36.048331 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:36.048398 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:36.048451 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:36.048510 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:36.048557 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:36.048588 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:36.048619 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:36.048658 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:36.048688 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:36.081349 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:36.084448 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.084548 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.084613 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.122313 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:36.215508 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0414 05:13:36.215953 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:36.216033 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:36.216082 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:36.216266 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:36.216267 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:36.216298 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:36.216379 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:36.222093 29933 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0414 05:13:36.222769 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:36.227097 29933 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0414 05:13:36.227512 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.227748 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.227976 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.228204 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.228887 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.230366 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.230504 29933 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0414 05:13:36.230945 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:13:36.291383 29933 target.go:112] Terraform output is in /tmp/test1914285879/out --- PASS: TestPrivateSharedSubnet (0.25s) === RUN TestPrivateSharedIP I0414 05:13:36.296025 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:36.296091 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:36.296145 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:36.296201 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:36.296293 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:36.296323 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:36.296354 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:36.296392 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:36.296425 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:36.324531 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:36.327600 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.327690 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.327748 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.364634 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:36.467280 29933 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0414 05:13:36.467656 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:36.469139 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:36.469171 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:36.469258 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:36.469877 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:36.470272 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:36.470537 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:36.473703 29933 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0414 05:13:36.473972 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:36.476602 29933 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0414 05:13:36.476930 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.477137 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.477418 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.477800 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.480250 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.481468 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.481804 29933 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0414 05:13:36.482854 29933 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0414 05:13:36.483197 29933 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0414 05:13:36.546810 29933 target.go:112] Terraform output is in /tmp/test1373984592/out I0414 05:13:36.551277 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:36.551347 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:36.551399 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:36.551455 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:36.551504 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:36.551533 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:36.551565 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:36.551606 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:36.551636 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:36.579832 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:36.582857 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.582952 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.583016 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.620957 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:36.724785 29933 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0414 05:13:36.725112 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:36.725114 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:36.725243 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:36.725266 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:36.725289 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:36.725446 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:36.725424 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:13:36.727969 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:36.728055 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:36.728224 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:36.732383 29933 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0414 05:13:36.733055 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:36.738360 29933 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0414 05:13:36.738712 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.738980 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.739168 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.739307 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.740019 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.741665 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:36.741822 29933 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0414 05:13:36.742165 29933 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0414 05:13:36.742316 29933 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0414 05:13:36.746234 29933 target.go:168] Cloudformation output is in /tmp/test2696107787/out --- PASS: TestPrivateSharedIP (0.47s) === RUN TestPrivateDns1 I0414 05:13:36.762452 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:36.762507 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:36.762557 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:36.762611 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:36.762702 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:36.762733 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:36.762766 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:36.762803 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:36.762833 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:36.792850 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.792928 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.792983 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.959111 29933 executor.go:111] Tasks: 0 done / 118 total; 47 can run W0414 05:13:36.959440 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:36.959473 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:36.959594 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:36.960088 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:36.965680 29933 executor.go:111] Tasks: 47 done / 118 total; 24 can run I0414 05:13:36.966303 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0414 05:13:36.966379 29933 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0414 05:13:36.966566 29933 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0414 05:13:36.966606 29933 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0414 05:13:36.966774 29933 executor.go:111] Tasks: 71 done / 118 total; 35 can run I0414 05:13:36.967206 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.967521 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.970421 29933 executor.go:111] Tasks: 106 done / 118 total; 5 can run W0414 05:13:36.970629 29933 api.go:53] PageSize not implemented W0414 05:13:36.970768 29933 api.go:53] PageSize not implemented W0414 05:13:36.970664 29933 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0414 05:13:36.970819 29933 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0414 05:13:36.973946 29933 executor.go:111] Tasks: 111 done / 118 total; 5 can run I0414 05:13:36.974175 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.974384 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.975412 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.977136 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:36.977260 29933 executor.go:111] Tasks: 116 done / 118 total; 2 can run I0414 05:13:36.977635 29933 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0414 05:13:37.056590 29933 target.go:112] Terraform output is in /tmp/test3880590895/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I0414 05:13:37.061702 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:37.061769 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:37.061824 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:37.061882 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:37.061927 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:37.061960 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:37.061998 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:37.062033 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:37.062062 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:37.096950 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.097058 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.097125 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.235377 29933 executor.go:111] Tasks: 0 done / 114 total; 45 can run W0414 05:13:37.235744 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:37.236259 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:37.236318 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:37.236334 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:37.238048 29933 executor.go:111] Tasks: 45 done / 114 total; 22 can run I0414 05:13:37.238489 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0414 05:13:37.238519 29933 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0414 05:13:37.238581 29933 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0414 05:13:37.238494 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:37.238594 29933 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0414 05:13:37.238692 29933 executor.go:111] Tasks: 67 done / 114 total; 35 can run I0414 05:13:37.239143 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.239390 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.244716 29933 executor.go:111] Tasks: 102 done / 114 total; 5 can run W0414 05:13:37.244817 29933 api.go:53] PageSize not implemented W0414 05:13:37.244972 29933 api.go:53] PageSize not implemented W0414 05:13:37.244854 29933 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0414 05:13:37.245009 29933 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0414 05:13:37.251680 29933 executor.go:111] Tasks: 107 done / 114 total; 5 can run I0414 05:13:37.251791 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.252021 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.253120 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.253218 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.253316 29933 executor.go:111] Tasks: 112 done / 114 total; 2 can run I0414 05:13:37.253642 29933 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0414 05:13:37.337088 29933 target.go:112] Terraform output is in /tmp/test649024134/out --- PASS: TestPrivateDns2 (0.28s) === RUN TestDiscoveryFeatureGate I0414 05:13:37.344909 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:37.344966 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:37.345015 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:37.345065 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:37.345106 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:37.345133 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:37.345163 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:37.345198 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:37.345225 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:37.370980 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0414 05:13:37.374088 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:37.374184 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.720046 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:38.796170 29933 executor.go:111] Tasks: 0 done / 91 total; 47 can run W0414 05:13:38.796716 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:38.796997 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:38.797081 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:38.797029 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:38.797108 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:38.797223 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:38.801631 29933 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0414 05:13:38.807107 29933 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0414 05:13:38.807664 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.808022 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.808357 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.810901 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.811023 29933 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0414 05:13:38.811322 29933 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0414 05:13:38.866162 29933 target.go:112] Terraform output is in /tmp/test2224021340/out --- PASS: TestDiscoveryFeatureGate (1.53s) === RUN TestVFSServiceAccountIssuerDiscovery I0414 05:13:38.870941 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:38.871007 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:38.871060 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:38.871117 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:38.871163 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:38.871192 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:38.871222 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:38.871260 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:38.871291 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:38.895831 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:38.898506 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.898594 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:38.936217 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:39.038727 29933 executor.go:111] Tasks: 0 done / 89 total; 46 can run W0414 05:13:39.039460 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:39.039537 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:39.039581 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:39.039674 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:39.039717 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:39.039777 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:39.041342 29933 executor.go:111] Tasks: 46 done / 89 total; 20 can run I0414 05:13:39.050905 29933 executor.go:111] Tasks: 66 done / 89 total; 21 can run I0414 05:13:39.051172 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:39.051454 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:39.052663 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:39.052980 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:39.053053 29933 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0414 05:13:39.053371 29933 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0414 05:13:39.097761 29933 target.go:112] Terraform output is in /tmp/test4175510655/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.23s) === RUN TestAWSLBController I0414 05:13:39.102538 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:39.102599 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:39.102659 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:39.102727 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:39.102840 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:39.102872 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:39.102916 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:39.102959 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:39.102993 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:39.127950 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:39.131146 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:39.131247 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:39.169305 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:40.372886 29933 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0414 05:13:40.373387 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:40.373350 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:40.373486 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:40.373719 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:40.373763 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:40.373900 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:40.382386 29933 executor.go:111] Tasks: 50 done / 95 total; 22 can run I0414 05:13:40.387057 29933 executor.go:111] Tasks: 72 done / 95 total; 21 can run I0414 05:13:40.387545 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:40.387858 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:40.388970 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:40.389931 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:40.390047 29933 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0414 05:13:40.390350 29933 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0414 05:13:40.446739 29933 target.go:112] Terraform output is in /tmp/test1783322692/out --- PASS: TestAWSLBController (1.36s) === RUN TestManyAddons I0414 05:13:40.465038 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:40.465109 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:40.465162 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:40.465219 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:40.465312 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:40.465342 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:40.465373 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:40.465413 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:40.465445 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:40.491292 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:40.494245 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:40.494328 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:40.533084 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:41.840921 29933 executor.go:111] Tasks: 0 done / 94 total; 51 can run W0414 05:13:41.841957 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:41.842996 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:41.845690 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:41.846318 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:41.846351 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:41.846447 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:41.859238 29933 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0414 05:13:41.864741 29933 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0414 05:13:41.865071 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:41.865319 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:41.866603 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:41.867681 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:41.867803 29933 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0414 05:13:41.868167 29933 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0414 05:13:41.902692 29933 target.go:112] Terraform output is in /tmp/test1795044647/out --- PASS: TestManyAddons (1.46s) === RUN TestManyAddonsCCMIRSA I0414 05:13:41.922211 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:41.922280 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:41.922331 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:41.922387 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:41.922477 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:41.922506 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:41.922537 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:41.922575 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:41.922605 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:41.954789 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:41.958686 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:41.958812 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:41.996289 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:43.365316 29933 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0414 05:13:43.365617 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:43.365655 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:43.365772 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:43.366150 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:43.366291 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:43.367020 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:43.387797 29933 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0414 05:13:43.394419 29933 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0414 05:13:43.394727 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:43.394984 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:43.396153 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:43.397671 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:43.397815 29933 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0414 05:13:43.398234 29933 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0414 05:13:43.468824 29933 target.go:112] Terraform output is in /tmp/test1533076071/out --- PASS: TestManyAddonsCCMIRSA (1.57s) === RUN TestCCM I0414 05:13:43.489594 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:43.489659 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:43.489715 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:43.489772 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:43.489870 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:43.489901 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:43.489932 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:43.489969 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:43.490000 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:43.514212 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:43.516853 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:43.516928 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:43.554204 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:44.845835 29933 executor.go:111] Tasks: 0 done / 95 total; 52 can run W0414 05:13:44.846026 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:44.846161 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:44.846488 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:44.846561 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:44.846658 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:44.846718 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:44.857370 29933 executor.go:111] Tasks: 52 done / 95 total; 19 can run I0414 05:13:44.866764 29933 executor.go:111] Tasks: 71 done / 95 total; 22 can run I0414 05:13:44.866983 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:44.867222 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:44.867941 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:44.868454 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:44.868521 29933 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0414 05:13:44.868867 29933 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0414 05:13:44.913293 29933 target.go:112] Terraform output is in /tmp/test667922161/out --- PASS: TestCCM (1.44s) === RUN TestExternalDNS I0414 05:13:44.933639 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:44.933707 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:44.933758 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:44.933814 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:44.933905 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:44.933938 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:44.933987 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:44.934025 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:44.934052 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:44.958692 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:44.961593 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:44.961692 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:44.998603 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:45.100579 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0414 05:13:45.100976 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:45.101275 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:45.101418 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:45.101642 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:45.101678 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:45.101771 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:45.103270 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:45.110099 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:45.110598 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.110845 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.111961 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.112959 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.113078 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:45.113344 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:45.170379 29933 target.go:112] Terraform output is in /tmp/test522965672/out I0414 05:13:45.174798 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:45.174867 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:45.174920 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:45.174980 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:45.175027 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:45.175056 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:45.175090 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:45.175132 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:45.175162 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:45.199500 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:45.202327 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.202427 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.239986 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:45.343020 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:45.343286 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:45.343311 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:45.343372 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:45.343437 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:45.343456 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:45.343818 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:45.343939 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:45.343992 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:45.344127 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:45.349595 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:45.355025 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:45.355430 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.355639 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.356838 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.358479 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.358604 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:45.358893 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:45.362312 29933 target.go:168] Cloudformation output is in /tmp/test3122603609/out --- PASS: TestExternalDNS (0.44s) === RUN TestExternalDNSIRSA I0414 05:13:45.372435 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:45.372489 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:45.372535 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:45.372590 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:45.372633 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:45.372661 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:45.372691 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:45.372726 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:45.372757 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:45.403294 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:45.406294 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.406412 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.444394 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:45.523306 29933 executor.go:111] Tasks: 0 done / 91 total; 47 can run W0414 05:13:45.523563 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:45.524054 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:45.524342 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:45.524366 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:45.524367 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:45.524425 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:45.526902 29933 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0414 05:13:45.532167 29933 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0414 05:13:45.532555 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.532810 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.535526 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.537553 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.537690 29933 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0414 05:13:45.538005 29933 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0414 05:13:45.583110 29933 target.go:112] Terraform output is in /tmp/test2284162749/out --- PASS: TestExternalDNSIRSA (0.21s) === RUN TestSharedSubnet I0414 05:13:45.586097 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:45.586134 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:45.586164 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:45.586198 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:45.586225 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:45.586242 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:45.586265 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:45.586285 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:45.586302 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:45.602844 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:45.605671 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.605775 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.643656 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:45.706446 29933 executor.go:111] Tasks: 0 done / 79 total; 43 can run W0414 05:13:45.706758 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:45.706984 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:45.707003 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:45.707057 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:45.707118 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:45.707178 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:45.708858 29933 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0414 05:13:45.709274 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:45.716234 29933 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0414 05:13:45.716641 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.716846 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.717728 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.719116 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.719235 29933 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0414 05:13:45.719525 29933 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0414 05:13:45.763640 29933 target.go:112] Terraform output is in /tmp/test4246303168/out --- PASS: TestSharedSubnet (0.18s) === RUN TestSharedVPC I0414 05:13:45.768364 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:45.768427 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:45.768480 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:45.768537 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:45.768580 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:45.768608 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:45.768639 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:45.768675 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:45.768705 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:45.795550 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:45.798363 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.798470 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.835760 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:45.913524 29933 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0414 05:13:45.914207 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:45.914308 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:45.914312 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:45.914256 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:45.914362 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:45.914392 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:45.915960 29933 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0414 05:13:45.916189 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:45.922618 29933 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0414 05:13:45.922894 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.923069 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.923703 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.925235 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:45.925358 29933 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0414 05:13:45.925714 29933 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0414 05:13:45.965415 29933 target.go:112] Terraform output is in /tmp/test2343428070/out --- PASS: TestSharedVPC (0.20s) === RUN TestExistingIAM I0414 05:13:45.967990 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:45.968025 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:45.968055 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:45.968086 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:45.968112 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:45.968130 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:45.968147 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:45.968169 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:45.968186 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:45.986398 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:45.988155 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:45.988208 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:45.988244 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:45.988278 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.025387 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:46.105746 29933 executor.go:111] Tasks: 0 done / 94 total; 48 can run W0414 05:13:46.106230 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:46.106326 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:46.106545 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.106577 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:46.106603 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.106622 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:46.106601 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:46.106685 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:46.113152 29933 executor.go:111] Tasks: 48 done / 94 total; 15 can run I0414 05:13:46.118210 29933 executor.go:111] Tasks: 63 done / 94 total; 27 can run I0414 05:13:46.118386 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.118547 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.118622 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.118862 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.120497 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.120801 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.123846 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.124650 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:46.124780 29933 executor.go:111] Tasks: 90 done / 94 total; 4 can run I0414 05:13:46.125205 29933 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0414 05:13:46.184844 29933 target.go:112] Terraform output is in /tmp/test3482249443/out --- PASS: TestExistingIAM (0.22s) === RUN TestPhaseNetwork I0414 05:13:46.189714 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:46.189780 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:46.189837 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:46.189898 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:46.189946 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:46.189976 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:46.190011 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:46.190049 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:46.190078 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:46.219784 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:46.223549 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.223891 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.224029 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.262469 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:46.360402 29933 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0414 05:13:46.360707 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:46.360899 29933 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0414 05:13:46.361395 29933 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0414 05:13:46.361825 29933 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0414 05:13:46.362021 29933 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0414 05:13:46.362158 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0414 05:13:46.370782 29933 target.go:112] Terraform output is in /tmp/test2766449561/out --- PASS: TestPhaseNetwork (0.18s) === RUN TestExternalLoadBalancer I0414 05:13:46.371735 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:46.371786 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:46.371834 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:46.371887 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:46.371996 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:46.372027 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:46.372060 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:46.372097 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:46.372125 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:46.392589 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:46.394181 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.394246 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.430867 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:46.532815 29933 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0414 05:13:46.533363 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.533495 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:46.533533 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:46.533690 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:46.533689 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:46.533502 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.535305 29933 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0414 05:13:46.542334 29933 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0414 05:13:46.542639 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.542819 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.543715 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.544130 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.544194 29933 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0414 05:13:46.544459 29933 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0414 05:13:46.600071 29933 target.go:112] Terraform output is in /tmp/test120287854/out I0414 05:13:46.604305 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:46.604373 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:46.604437 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:46.604506 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:46.604570 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:46.604602 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:46.604644 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:46.604687 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:46.604725 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:46.629531 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:46.632625 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.632735 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.670585 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:46.771219 29933 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0414 05:13:46.771748 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:46.771788 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0414 05:13:46.771977 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:46.771881 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:46.773497 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:46.773699 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:46.773725 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:46.773847 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W0414 05:13:46.774642 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:46.778910 29933 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0414 05:13:46.783940 29933 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0414 05:13:46.784413 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.784692 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.785979 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.787476 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.787617 29933 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0414 05:13:46.788029 29933 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0414 05:13:46.791241 29933 target.go:168] Cloudformation output is in /tmp/test1114213055/out --- PASS: TestExternalLoadBalancer (0.43s) === RUN TestPhaseIAM integration_test.go:633: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:641: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0414 05:13:46.801356 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:46.801411 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:46.801460 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:46.801512 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:46.801594 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:46.801624 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:46.801656 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:46.801692 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:46.801722 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:46.839191 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:46.842326 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.842427 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.842491 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.842563 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.842619 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.842670 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.842721 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.879660 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:46.964407 29933 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0414 05:13:46.964790 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.964856 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:46.964946 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:46.965065 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.965154 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:46.965170 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:46.965222 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:46.965360 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:46.966739 29933 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0414 05:13:46.973987 29933 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0414 05:13:46.974287 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.974580 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.974871 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.975115 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.975706 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.976757 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.977266 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.977756 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:46.977882 29933 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0414 05:13:46.978337 29933 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0414 05:13:47.042694 29933 target.go:112] Terraform output is in /tmp/test4285654209/out I0414 05:13:47.045316 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:47.045353 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:47.045382 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:47.045413 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:47.045436 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:47.045452 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:47.045470 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:47.045492 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:47.045507 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:47.072596 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:47.076286 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.076414 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.076483 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.076540 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.076596 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.076644 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.076694 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.114173 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:47.195692 29933 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0414 05:13:47.195961 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:47.196245 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:47.196408 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:47.196544 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:47.196638 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:47.196603 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0414 05:13:47.196722 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:47.196746 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:47.196737 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:47.196869 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:47.196865 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0414 05:13:47.198273 29933 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0414 05:13:47.207681 29933 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0414 05:13:47.207966 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.208157 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.208446 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.208678 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.210598 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.210997 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.211373 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.211902 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.212034 29933 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0414 05:13:47.212446 29933 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0414 05:13:47.217833 29933 target.go:168] Cloudformation output is in /tmp/test2927810233/out --- PASS: TestMixedInstancesASG (0.44s) === RUN TestMixedInstancesSpotASG I0414 05:13:47.237617 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:47.237673 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:47.237724 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:47.237778 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:47.237873 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:47.237903 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:47.237934 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:47.237971 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:47.238002 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:47.269915 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:47.272727 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.272811 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.272865 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.272934 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.272985 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.273030 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.273070 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.310807 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:47.402541 29933 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0414 05:13:47.403041 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:47.403085 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:47.403485 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:47.403610 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:47.403485 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:47.403935 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:47.404039 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:47.404686 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:47.407845 29933 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0414 05:13:47.413013 29933 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0414 05:13:47.413370 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.413637 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.414662 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.414918 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.418309 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.418642 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.419191 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.419687 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.419831 29933 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0414 05:13:47.420257 29933 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0414 05:13:47.490208 29933 target.go:112] Terraform output is in /tmp/test4135159150/out I0414 05:13:47.493604 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:47.493649 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:47.493686 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:47.493724 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:47.493758 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:47.493779 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:47.493803 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:47.493828 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:47.493849 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:47.520481 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:47.522792 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.522864 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.522909 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.523004 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.523043 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.523078 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.523113 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.561118 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:47.656634 29933 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0414 05:13:47.656932 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:47.657045 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:47.657351 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:47.658713 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:47.658822 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:47.658833 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:47.658880 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:47.658897 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" },{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" }] }] } I0414 05:13:47.658944 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:47.659115 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0414 05:13:47.659790 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:47.662327 29933 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0414 05:13:47.667761 29933 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0414 05:13:47.668001 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.668226 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.668406 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.668635 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.669193 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.669606 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.672383 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.675029 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:47.675193 29933 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0414 05:13:47.675518 29933 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0414 05:13:47.680068 29933 target.go:168] Cloudformation output is in /tmp/test3225866138/out --- PASS: TestMixedInstancesSpotASG (0.46s) === RUN TestContainerd I0414 05:13:47.698529 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:47.698574 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:47.698662 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:47.698721 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:47.698765 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:47.698799 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:47.698836 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:47.698873 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:47.698903 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:47.723510 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:47.726221 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.726304 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.764531 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:47.861164 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0414 05:13:47.861697 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:47.861890 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0414 05:13:47.861921 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:47.862566 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:47.862990 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:47.863022 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:47.863144 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0414 05:13:47.861928 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:47.863826 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:47.868516 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:47.871583 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:47.871986 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.872143 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.873311 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.873449 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.873576 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:47.873782 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:47.876791 29933 target.go:168] Cloudformation output is in /tmp/test458845889/out --- PASS: TestContainerd (0.19s) === RUN TestContainerdCustom I0414 05:13:47.887749 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:47.887804 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:47.887846 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:47.887899 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:47.887977 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:47.888007 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:47.888038 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:47.888076 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:47.888105 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:47.916783 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:47.919830 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.919932 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:47.956552 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:48.033498 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0414 05:13:48.033984 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:48.034107 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:48.034323 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:48.034375 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0414 05:13:48.034421 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:48.034436 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:48.034415 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:48.034553 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:48.034493 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0414 05:13:48.036602 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:48.044780 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:48.045148 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.045362 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.046399 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.047721 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.047855 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:48.048218 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:48.051389 29933 target.go:168] Cloudformation output is in /tmp/test115947695/out --- PASS: TestContainerdCustom (0.17s) === RUN TestDockerCustom I0414 05:13:48.062499 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:48.062550 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:48.062593 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:48.062645 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:48.062720 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:48.062749 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:48.062782 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:48.062817 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:48.062846 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:48.087833 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:48.090710 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.090794 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.128207 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:48.237117 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:48.237459 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:48.237498 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:48.237614 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:48.237751 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:48.237958 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0414 05:13:48.238029 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:48.237994 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:48.238158 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:48.238111 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I0414 05:13:48.240171 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:48.242943 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:48.243102 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.243345 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.244101 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.245627 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:48.245772 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:48.245986 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:48.248834 29933 target.go:168] Cloudformation output is in /tmp/test27887410/out --- PASS: TestDockerCustom (0.20s) === RUN TestAPIServerNodes I0414 05:13:48.261843 29933 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0414 05:13:48.262195 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:48.262287 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:48.262423 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:48.262479 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:48.262524 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:48.262556 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:48.262588 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:48.262624 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:48.262651 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:48.290102 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0414 05:13:48.293094 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.293185 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.293251 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.334284 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:48.497943 29933 executor.go:111] Tasks: 0 done / 98 total; 48 can run W0414 05:13:48.498255 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:48.498329 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:48.498430 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:48.498735 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:48.498851 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0414 05:13:48.498860 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:48.498869 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:48.498917 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:48.499039 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:48.498955 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0414 05:13:48.501092 29933 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0414 05:13:48.509698 29933 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0414 05:13:48.509932 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.510034 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.510268 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.510575 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.511330 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.511791 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.511870 29933 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0414 05:13:48.512187 29933 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0414 05:13:48.515708 29933 target.go:168] Cloudformation output is in /tmp/test3873754660/out I0414 05:13:48.529997 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:48.530052 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:48.530099 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:48.530157 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:48.530201 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:48.530231 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:48.530264 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:48.530299 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:48.530327 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:48.561305 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0414 05:13:48.564289 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.564385 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.564452 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.601422 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:48.716483 29933 executor.go:111] Tasks: 0 done / 98 total; 48 can run W0414 05:13:48.717000 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:48.717366 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0414 05:13:48.717414 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:48.717435 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:48.717397 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:48.717476 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:48.717512 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:48.719100 29933 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0414 05:13:48.722978 29933 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0414 05:13:48.723391 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.723673 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.723904 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.727161 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.727435 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.728513 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.728646 29933 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0414 05:13:48.728946 29933 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0414 05:13:48.789372 29933 target.go:112] Terraform output is in /tmp/test2599517782/out I0414 05:13:48.793498 29933 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.53s) === RUN TestNTHQueueProcessor I0414 05:13:48.793742 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:48.793783 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:48.793822 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:48.793867 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:48.793942 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:48.793985 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:48.794014 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:48.794047 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:48.794073 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:48.809319 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0414 05:13:48.811580 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:48.811654 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.112364 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:50.209260 29933 executor.go:111] Tasks: 0 done / 98 total; 46 can run W0414 05:13:50.209760 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:50.209957 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:50.210116 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:50.210179 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:50.210285 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:50.210404 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:50.211733 29933 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0414 05:13:50.214761 29933 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0414 05:13:50.214926 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.215075 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.215956 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.216135 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.216245 29933 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0414 05:13:50.216507 29933 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0414 05:13:50.216672 29933 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0414 05:13:50.248226 29933 target.go:112] Terraform output is in /tmp/test3502844243/out I0414 05:13:50.256722 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:50.256785 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:50.256833 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:50.256885 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:50.256930 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:50.256960 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:50.256993 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:50.257028 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:50.257055 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:50.281822 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0414 05:13:50.284722 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.284816 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.324616 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0414 05:13:50.446162 29933 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0414 05:13:50.446366 29933 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0414 05:13:50.446395 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:50.446476 29933 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0414 05:13:50.446793 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0414 05:13:50.446860 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0414 05:13:50.448342 29933 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0414 05:13:50.448478 29933 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0414 05:13:50.448385 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:50.448554 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:13:50.452947 29933 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0414 05:13:50.458621 29933 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0414 05:13:50.458929 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.459169 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.459823 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.461790 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.461932 29933 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0414 05:13:50.462146 29933 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0414 05:13:50.462301 29933 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0414 05:13:50.465951 29933 target.go:168] Cloudformation output is in /tmp/test517049840/out --- PASS: TestNTHQueueProcessor (1.68s) === RUN TestCustomIRSA I0414 05:13:50.476574 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:50.476629 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:50.476670 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:50.476722 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:50.476804 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:50.476833 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:50.476863 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:50.476900 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:50.476927 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:50.498068 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0414 05:13:50.499833 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.499891 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:50.536997 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:51.718882 29933 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0414 05:13:51.719615 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:51.719741 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:51.719806 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:51.719833 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:51.719942 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:51.720026 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:51.731531 29933 executor.go:111] Tasks: 51 done / 99 total; 25 can run I0414 05:13:51.735092 29933 executor.go:111] Tasks: 76 done / 99 total; 21 can run I0414 05:13:51.735314 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.736451 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.737542 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.740219 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.740360 29933 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0414 05:13:51.740645 29933 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0414 05:13:51.800731 29933 target.go:112] Terraform output is in /tmp/test3846343911/out --- PASS: TestCustomIRSA (1.35s) === RUN TestCustomIRSA119 I0414 05:13:51.823836 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:51.823909 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:51.823981 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:51.824039 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:51.824086 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:51.824115 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:51.824149 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:51.824187 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:51.824216 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:51.852385 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0414 05:13:51.855815 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.855914 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.896666 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:51.970035 29933 executor.go:111] Tasks: 0 done / 97 total; 49 can run W0414 05:13:51.970304 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:51.970679 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:51.970739 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:51.970767 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:51.970870 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0414 05:13:51.970981 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:51.973533 29933 executor.go:111] Tasks: 49 done / 97 total; 25 can run I0414 05:13:51.978838 29933 executor.go:111] Tasks: 74 done / 97 total; 21 can run I0414 05:13:51.979235 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.979341 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.980504 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.980588 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:51.980679 29933 executor.go:111] Tasks: 95 done / 97 total; 2 can run I0414 05:13:51.981002 29933 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0414 05:13:52.039417 29933 target.go:112] Terraform output is in /tmp/test3355109339/out --- PASS: TestCustomIRSA119 (0.22s) === RUN TestClusterNameDigit I0414 05:13:52.044327 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:52.044392 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:52.044444 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:52.044502 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:52.044546 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:52.044575 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:52.044608 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:52.044646 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:52.044674 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:52.063691 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0414 05:13:52.065376 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.065432 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.103435 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:52.171958 29933 executor.go:111] Tasks: 0 done / 94 total; 48 can run I0414 05:13:52.172195 29933 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0414 05:13:52.172212 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:52.172271 29933 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0414 05:13:52.172487 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target W0414 05:13:52.172568 29933 warmpool.go:124] ASG warm pool is not supported by the terraform target I0414 05:13:52.172915 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:52.175365 29933 executor.go:111] Tasks: 48 done / 94 total; 23 can run I0414 05:13:52.183060 29933 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0414 05:13:52.183363 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.183601 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.184672 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.184766 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.184949 29933 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0414 05:13:52.185357 29933 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0414 05:13:52.244420 29933 target.go:112] Terraform output is in /tmp/test1466802539/out --- PASS: TestClusterNameDigit (0.21s) === RUN TestLifecycleMinimalAWS I0414 05:13:52.249497 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:52.249579 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:52.249635 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:52.249694 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:52.249741 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:52.249770 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:52.249805 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:52.249844 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:52.249872 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:52.249938 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0414 05:13:52.273387 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:52.276216 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.276318 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0414 05:13:52.314997 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:52.413267 29933 iamrole.go:91] ListRoles: { } I0414 05:13:52.414770 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:52.415077 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:52.415072 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.415116 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0414 05:13:52.415282 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0414 05:13:52.415318 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:52.415367 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:52.415352 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0414 05:13:52.415353 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.415476 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.415503 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0414 05:13:52.415578 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.415624 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0414 05:13:52.415680 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" }] } I0414 05:13:52.415714 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0414 05:13:52.417464 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:13:52.417640 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.417890 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.418052 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:52.418205 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:13:52.418286 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:52.418382 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:52.418644 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:52.420377 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:52.420482 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:52.420658 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:52.420938 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:52.421126 29933 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0414 05:13:52.421055 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.421166 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0414 05:13:52.421196 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0414 05:13:52.421404 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:52.421393 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0414 05:13:52.421509 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:52.421566 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:52.421537 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0414 05:13:52.421613 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] }] } I0414 05:13:52.421776 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:13:52.421802 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.421881 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0414 05:13:52.421990 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:52.422090 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0414 05:13:52.422129 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0414 05:13:52.422355 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:52.422483 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:52.422587 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:52.422688 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:52.422749 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:52.422793 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:52.422853 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.422938 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:52.422996 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:52.423044 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0414 05:13:52.423129 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:52.423231 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:52.423283 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:13:52.423255 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0414 05:13:52.424068 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:52.424214 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" }] }] } I0414 05:13:52.424362 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.424552 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:52.424637 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0414 05:13:52.424912 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.425066 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.425220 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.425333 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.425439 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0414 05:13:52.425596 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.425719 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.425844 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.426011 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.426187 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I0414 05:13:52.426351 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" }] }] } I0414 05:13:52.426499 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.426631 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.426767 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:52.426823 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.426917 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.426996 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.427188 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:52.427246 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0414 05:13:52.427309 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.427430 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0414 05:13:52.429329 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:52.429600 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:52.429665 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:52.429704 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:52.429770 29933 dns.go:238] Pre-creating DNS records I0414 05:13:52.429802 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:52.436143 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:52.439003 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.439097 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0414 05:13:52.477949 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:52.569153 29933 iamrole.go:91] ListRoles: { } I0414 05:13:52.570088 29933 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0414 05:13:52.570334 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.570602 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0414 05:13:52.570725 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:52.570656 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:52.570756 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:52.570883 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:52.570959 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:52.571071 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:52.571263 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.571495 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.573816 29933 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0414 05:13:52.574089 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0414 05:13:52.573997 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.574203 29933 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0414 05:13:52.574268 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0414 05:13:52.574674 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0414 05:13:52.574930 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:52.575134 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:52.575292 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.575525 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.575669 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:52.575728 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:52.576136 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:52.579162 29933 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0414 05:13:52.579271 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.579376 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.579499 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.579640 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.580053 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.580320 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.580485 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:52.582259 29933 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0414 05:13:52.582461 29933 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0414 05:13:52.585578 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:52.585644 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:52.585682 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:52.585717 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:52.585756 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:52.585791 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:52.585824 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:52.585857 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:52.585894 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:13:52.585927 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:52.585966 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:52.586007 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:52.586041 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:52.586076 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:52.586114 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:52.586146 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:52.586181 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:52.586215 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:52.586254 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:52.586287 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:52.586319 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:52.586351 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:52.586387 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:52.586424 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:52.586460 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:52.588169 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:52.588196 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:52.588234 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588285 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0414 05:13:52.588343 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588402 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588448 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588504 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588551 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588591 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0414 05:13:52.588640 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588693 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0414 05:13:52.588732 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0414 05:13:52.588778 29933 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0414 05:13:52.588808 29933 api.go:53] PageSize not implemented I0414 05:13:52.588828 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:52.588860 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:52.588879 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:52.588910 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:52.588929 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:52.588986 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:52.589017 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:52.589075 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:52.589130 29933 iamrole.go:91] ListRoles: { } I0414 05:13:52.589196 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:52.589241 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:52.589274 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:52.589848 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0414 05:13:52.589931 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:52.589981 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0414 05:13:52.590035 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.minimal.example.com ok I0414 05:13:52.590166 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0414 05:13:52.590219 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0414 05:13:52.590294 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0414 05:13:52.590196 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } I0414 05:13:52.590132 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok W0414 05:13:52.590355 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:52.590401 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:13:52.590445 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0414 05:13:52.590572 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:52.590595 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:52.590657 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0414 05:13:52.590725 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0414 05:13:52.590773 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:13:52.590807 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0414 05:13:52.590861 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0414 05:13:52.590893 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal.example.com I0414 05:13:52.590903 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0414 05:13:52.590924 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0414 05:13:52.590950 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0414 05:13:52.590974 29933 iamrole.go:182] ListAttachedRolePolicies: masters.minimal.example.com I0414 05:13:52.590982 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0414 05:13:52.590998 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0414 05:13:52.590967 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:52.591101 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:13:52.591189 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:13:52.591270 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:13:52.591531 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.34s) === RUN TestLifecycleMinimalOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0414 05:13:52.599485 29933 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:52.664125 29933 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0414 05:13:52.776605 29933 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0414 05:13:52.785063 29933 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0414 05:13:52.812236 29933 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0414 05:13:52.815113 29933 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0414 05:13:52.817957 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:52.818312 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:52.818469 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0414 05:13:52.822242 29933 executor.go:111] Tasks: 97 done / 97 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:52.864629 29933 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0414 05:13:52.987838 29933 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0414 05:13:52.994936 29933 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0414 05:13:53.003552 29933 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0414 05:13:53.006042 29933 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0414 05:13:53.007882 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:53.007932 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:53.008167 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:53.008434 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:53.008635 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:53.009005 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0414 05:13:53.010115 29933 executor.go:111] Tasks: 97 done / 97 total; 0 can run Instance:05be899d-369a-465c-9257-0d20bd721544 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:8cfc81ba-705b-486d-9cf3-655f2062131f ok Network:76864cb9-2130-4c35-82dc-781355e5d41f ok Port:94ad0d9b-de30-477d-8e56-2932a04d954e ok Subnet:bb03c0b9-802c-4238-9611-7730307c6d35 ok SecurityGroup:ed9f8a45-c31f-451a-88b6-a3696c41316e ok ServerGroup:8a68bba0-fdfb-4d5b-ac68-e4bd1f40290c ok Port:69a916b5-ffaf-4b5e-b332-1b53634b0c13 ok Instance:1e2decb1-cb89-45a4-831c-1f537342b41f ok SecurityGroup:cf31ad76-0165-4c8b-aeae-0ead80863a02 ok SecurityGroup:bcc251c1-6a6c-4bbf-8c1a-69e6fd19ecfd ok ServerGroup:691b6bd8-7756-4af1-a0e3-5d30c326b840 ok Instance:7224b6f3-d0db-4eac-8711-71c7d2db9fb2 ok --- PASS: TestLifecycleMinimalOpenstack (1.52s) === RUN TestLifecycleMinimalGCE I0414 05:13:54.114660 29933 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true lifecycle_integration_test.go:488: running lifecycle test for cluster minimal-gce.example.com I0414 05:13:54.115618 29933 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0414 05:13:54.314193 29933 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0414 05:13:54.314288 29933 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0414 05:13:54.314422 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0414 05:13:54.314464 29933 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0414 05:13:54.314488 29933 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0414 05:13:54.314511 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:54.314582 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:54.314605 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:54.316192 29933 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0414 05:13:54.323079 29933 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0414 05:13:54.326293 29933 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0414 05:13:54.331328 29933 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0414 05:13:54.331523 29933 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0414 05:13:54.331615 29933 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0414 05:13:54.491766 29933 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0414 05:13:54.491854 29933 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0414 05:13:54.491971 29933 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0414 05:13:54.492011 29933 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0414 05:13:54.492034 29933 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0414 05:13:54.492055 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:54.492128 29933 autoscalinggroup.go:160] VMs using Service Account: default I0414 05:13:54.492150 29933 autoscalinggroup.go:168] gsa: default I0414 05:13:54.493622 29933 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0414 05:13:54.496122 29933 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0414 05:13:54.501143 29933 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0414 05:13:54.505835 29933 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0414 05:13:54.506040 29933 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0414 05:13:54.507666 29933 gce.go:93] Scanning zones: [us-test1-a] Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1649913234 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1649913234 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.39s) === RUN TestLifecycleFloatingIPOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0414 05:13:54.515072 29933 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:54.578312 29933 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0414 05:13:54.735167 29933 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0414 05:13:54.743528 29933 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0414 05:13:54.780790 29933 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0414 05:13:54.785435 29933 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0414 05:13:54.786881 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:54.787173 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:54.787639 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0414 05:13:54.790272 29933 executor.go:111] Tasks: 99 done / 99 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:54.831835 29933 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0414 05:13:54.992996 29933 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0414 05:13:54.996512 29933 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0414 05:13:55.010043 29933 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0414 05:13:55.012529 29933 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0414 05:13:55.015932 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:55.016111 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:55.016604 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:55.017023 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:55.018084 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0414 05:13:55.018223 29933 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0414 05:13:55.020286 29933 executor.go:111] Tasks: 99 done / 99 total; 0 can run Router-IF:56247b56-2324-4511-8f13-d20a29cb5c18 ok Network:a349baa8-a963-4a6a-be8f-cfadf452754a ok Instance:a7b2521d-b751-419c-bcb4-3e76bb8a579f ok Instance:b4335ce8-80b6-4d97-83b4-c0dd74cec8d5 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok ServerGroup:d465b8cf-b033-46f1-8856-67b4ebf88cc6 ok SecurityGroup:2c8f964c-9e2b-4f57-a2a2-f6e8da1d285d ok ServerGroup:ed238bd3-db4b-413d-a6c0-87df2f4fc525 ok Port:fb56cd40-660d-4773-acb1-c0f408768375 ok Subnet:56247b56-2324-4511-8f13-d20a29cb5c18 ok Router:b0b231f5-bd4b-4862-bd44-fe4a0b913366 ok Port:69ffe455-91fc-4c70-a8f3-8ce984c533cc ok Port:ebd3d124-f406-44ff-ab90-2a651b8b0975 ok Port:51cb9410-4876-4f07-afda-00d7fad39799 ok SecurityGroup:aaac4ede-4134-4ed7-8117-22d7e18578a2 ok Instance:c3973c13-6c39-400c-a340-f7b7424493be ok SecurityGroup:c7bffda2-8724-47d1-93c4-96c3c591b264 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.62s) === RUN TestLifecyclePrivateCalico I0414 05:13:56.132654 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:56.132687 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:56.132717 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:56.132747 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:56.132772 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:56.132790 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:56.132807 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:56.132827 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:56.132841 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:56.132880 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0414 05:13:56.156219 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:56.159788 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.159896 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.159963 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.198509 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:56.489490 29933 iamrole.go:91] ListRoles: { } I0414 05:13:56.491821 29933 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0414 05:13:56.492044 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:56.492107 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:56.492244 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.492393 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0414 05:13:56.492463 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0414 05:13:56.492518 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.492555 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.492571 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:56.492652 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0414 05:13:56.492687 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:56.492800 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:56.492773 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0414 05:13:56.492845 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.492940 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0414 05:13:56.492963 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.493004 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.493060 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:56.493100 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:56.493132 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.493202 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:56.493234 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.493372 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.493353 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.493644 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.493806 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:13:56.493872 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:13:56.493926 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:56.494031 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:56.494617 29933 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0414 05:13:56.494750 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0414 05:13:56.494780 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.494846 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0414 05:13:56.494897 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.495012 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0414 05:13:56.495007 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0414 05:13:56.495060 29933 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0414 05:13:56.495000 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0414 05:13:56.495102 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0414 05:13:56.495191 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0414 05:13:56.495257 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0414 05:13:56.495254 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:56.495995 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0414 05:13:56.495965 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.496021 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0414 05:13:56.496298 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0414 05:13:56.496566 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.496768 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0414 05:13:56.496894 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0414 05:13:56.497029 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0414 05:13:56.497211 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:56.497362 29933 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0414 05:13:56.497558 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0414 05:13:56.497785 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:56.497871 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.498048 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:56.498385 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:56.498372 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:13:56.499952 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0414 05:13:56.500120 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0414 05:13:56.500226 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.500311 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:56.500399 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0414 05:13:56.500512 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:56.500569 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:56.500639 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:13:56.500685 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:56.500714 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:56.500754 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:56.500782 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:56.500843 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:13:56.500872 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:13:56.500899 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:56.500935 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:56.500961 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:56.500988 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:56.501021 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0414 05:13:56.501068 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:56.501099 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:56.500727 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0414 05:13:56.501124 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:56.501179 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:13:56.501205 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:13:56.501229 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:56.503460 29933 executor.go:111] Tasks: 76 done / 119 total; 37 can run W0414 05:13:56.503656 29933 api.go:53] PageSize not implemented W0414 05:13:56.503689 29933 api.go:53] PageSize not implemented I0414 05:13:56.503670 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.503768 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0414 05:13:56.503796 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.503843 29933 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.503892 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0414 05:13:56.503912 29933 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.503867 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:56.503925 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0414 05:13:56.504045 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0414 05:13:56.504091 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:56.504108 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.504236 29933 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0414 05:13:56.504286 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.504307 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0414 05:13:56.504243 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.504356 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0414 05:13:56.504492 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.504711 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.504858 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.505057 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:56.505202 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:56.505397 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.505772 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0414 05:13:56.505953 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.506042 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0414 05:13:56.506147 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I0414 05:13:56.506395 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.506599 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0414 05:13:56.506738 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" }] }] } I0414 05:13:56.506909 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:13:56.507067 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.507231 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:56.507324 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" }] }] } I0414 05:13:56.507492 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0414 05:13:56.507651 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:56.507692 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.507801 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.507931 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.508053 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:13:56.508158 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.508294 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.508390 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0414 05:13:56.508433 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.508506 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0414 05:13:56.508623 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.508754 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0414 05:13:56.508885 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.509015 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.509141 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0414 05:13:56.509178 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.509224 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.509358 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0414 05:13:56.509437 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.511832 29933 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0414 05:13:56.511950 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:56.511963 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.512060 29933 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:56.512119 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:56.512205 29933 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0414 05:13:56.512305 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0414 05:13:56.512356 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:56.512398 29933 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0414 05:13:56.512466 29933 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0414 05:13:56.512534 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.512661 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0414 05:13:56.512733 29933 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0414 05:13:56.512785 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:56.512818 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:56.512875 29933 dns.go:238] Pre-creating DNS records I0414 05:13:56.512900 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:56.518909 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:56.521887 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.521984 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.522049 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.559207 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:56.843308 29933 iamrole.go:91] ListRoles: { } I0414 05:13:56.844649 29933 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0414 05:13:56.844932 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.844972 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:56.845078 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0414 05:13:56.846432 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:56.846507 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:56.847057 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.847181 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:56.847309 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:56.847476 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.847853 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:56.851139 29933 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0414 05:13:56.851308 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0414 05:13:56.851410 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0414 05:13:56.851448 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0414 05:13:56.851460 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:56.851596 29933 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0414 05:13:56.851519 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0414 05:13:56.851768 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:13:56.851848 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0414 05:13:56.852003 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:13:56.852055 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.852207 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:56.852248 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.852373 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.852501 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:56.852620 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:56.852739 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0414 05:13:56.852935 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:56.853006 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.853144 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0414 05:13:56.853280 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:56.853366 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.853600 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:56.857242 29933 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0414 05:13:56.857354 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0414 05:13:56.857438 29933 api.go:53] PageSize not implemented W0414 05:13:56.857475 29933 api.go:53] PageSize not implemented I0414 05:13:56.857439 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.857496 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.857637 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0414 05:13:56.857610 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:56.857708 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0414 05:13:56.857815 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.857879 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.857903 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0414 05:13:56.857811 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.858050 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:13:56.858092 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0414 05:13:56.858119 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0414 05:13:56.858276 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.858350 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.858492 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:56.858561 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.858648 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:56.858823 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.858980 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:56.859981 29933 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0414 05:13:56.860034 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:56.860079 29933 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0414 05:13:56.860096 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W0414 05:13:56.860131 29933 api.go:53] PageSize not implemented I0414 05:13:56.860186 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.860163 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.860245 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0414 05:13:56.860336 29933 api.go:53] PageSize not implemented I0414 05:13:56.860351 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.860458 29933 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0414 05:13:56.860495 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:56.860556 29933 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0414 05:13:56.862501 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:56.862538 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:56.862560 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:56.862581 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:56.862608 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0414 05:13:56.862630 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:56.862649 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:56.862669 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:56.862688 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:56.862709 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:13:56.862729 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:56.862753 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:56.862779 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0414 05:13:56.862805 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0414 05:13:56.862827 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:56.862849 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:56.862871 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:56.862891 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:56.862910 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:56.862931 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:56.862953 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:56.862975 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:56.862995 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:13:56.863016 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:13:56.863037 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:56.863059 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:56.863079 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:56.863100 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:56.863119 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:56.863137 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:56.863160 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:56.863183 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:56.863204 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:56.864093 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:56.864108 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:56.864129 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864164 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0414 05:13:56.864211 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864247 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864273 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864303 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864328 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864355 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0414 05:13:56.864388 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864421 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:56.864436 29933 natgateway.go:117] DescribeNatGateways: { } I0414 05:13:56.864456 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0414 05:13:56.864479 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0414 05:13:56.864505 29933 api.go:53] PageSize not implemented I0414 05:13:56.864514 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0414 05:13:56.864544 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:56.864560 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:56.864570 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:56.864582 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:56.864592 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:56.864628 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:56.864642 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:56.864666 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:56.864691 29933 iamrole.go:91] ListRoles: { } I0414 05:13:56.864716 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:56.864744 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:56.864768 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0414 05:13:56.864791 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:13:56.864809 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0414 05:13:56.865170 29933 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0414 05:13:56.865222 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0414 05:13:56.865249 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } autoscaling-config:lt-3 ok I0414 05:13:56.865289 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0414 05:13:56.865290 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } autoscaling-group:nodes.privatecalico.example.com ok W0414 05:13:56.865356 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:56.865329 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } load-balancer:api-privatecalico-example-0uch4k ok I0414 05:13:56.865370 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0414 05:13:56.865381 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0414 05:13:56.865396 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:bastion.privatecalico.example.com ok I0414 05:13:56.865419 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0414 05:13:56.865458 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0414 05:13:56.865435 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:56.865277 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:56.865491 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok keypair:key-1 ok I0414 05:13:56.865474 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0414 05:13:56.865519 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0414 05:13:56.865566 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok W0414 05:13:56.865584 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:56.865594 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0414 05:13:56.865571 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok iam-instance-profile:bastions.privatecalico.example.com ok I0414 05:13:56.865612 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:13:56.865643 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0414 05:13:56.865672 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:56.865701 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0414 05:13:56.865822 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:56.865834 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0414 05:13:56.865888 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:56.865923 29933 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0414 05:13:56.865955 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:13:56.865939 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0414 05:13:56.866022 29933 iamrole.go:182] ListAttachedRolePolicies: bastions.privatecalico.example.com I0414 05:13:56.865987 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0414 05:13:56.866046 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:56.866055 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } I0414 05:13:56.866042 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } security-group:sg-4 ok I0414 05:13:56.866081 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0414 05:13:56.866082 29933 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0414 05:13:56.866110 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:56.866187 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0414 05:13:56.866237 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:56.866196 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0414 05:13:56.866244 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:56.866267 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.privatecalico.example.com I0414 05:13:56.866283 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0414 05:13:56.866317 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0414 05:13:56.866342 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0414 05:13:56.866374 29933 iamrole.go:182] ListAttachedRolePolicies: masters.privatecalico.example.com I0414 05:13:56.866382 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0414 05:13:56.866402 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0414 05:13:56.866447 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:13:56.866481 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0414 05:13:56.866524 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:13:56.866568 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:13:56.866722 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.73s) === RUN TestLifecyclePrivateKopeio I0414 05:13:56.867002 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:56.867029 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:56.867052 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:56.867079 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:56.867103 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:56.867121 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:56.867137 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:56.867158 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:56.867172 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:56.867205 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0414 05:13:56.886566 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:56.888267 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:56.888321 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:56.888357 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:56.926829 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.001481 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.002824 29933 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0414 05:13:57.003133 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" }] } I0414 05:13:57.003263 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.003498 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:57.003533 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.003671 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:57.003716 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0414 05:13:57.003742 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.003773 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:57.003797 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.003824 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.003874 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0414 05:13:57.003888 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.003940 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.004026 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:57.004115 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:57.004096 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.004148 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.004225 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0414 05:13:57.004291 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.004290 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.004330 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.004435 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.004501 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:57.004529 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.004635 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:13:57.004657 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:13:57.004672 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:57.004702 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.006009 29933 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0414 05:13:57.006212 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.006245 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0414 05:13:57.006377 29933 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0414 05:13:57.006345 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.006485 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0414 05:13:57.006511 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:13:57.006479 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.006548 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0414 05:13:57.006701 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0414 05:13:57.006780 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0414 05:13:57.006714 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0414 05:13:57.006838 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:57.006857 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0414 05:13:57.006909 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:57.006947 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0414 05:13:57.006978 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.007045 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0414 05:13:57.007130 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0414 05:13:57.007356 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0414 05:13:57.008674 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0414 05:13:57.008928 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0414 05:13:57.009011 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0414 05:13:57.009121 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] }] } I0414 05:13:57.009423 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.009424 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:13:57.009690 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.010026 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.010283 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.012036 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0414 05:13:57.012114 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0414 05:13:57.012176 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.012240 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:13:57.012262 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:57.012288 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0414 05:13:57.012341 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:57.012388 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.012348 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0414 05:13:57.012438 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:57.012514 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0414 05:13:57.012585 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:57.012641 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0414 05:13:57.012621 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0414 05:13:57.012644 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0414 05:13:57.012772 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:13:57.012837 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:13:57.012885 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.012936 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:57.012977 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:57.013069 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:13:57.013117 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:13:57.013155 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.013233 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0414 05:13:57.013334 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.013390 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:57.013456 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.013499 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:57.013559 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:57.013608 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:57.013662 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0414 05:13:57.013719 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:57.013776 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0414 05:13:57.013831 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0414 05:13:57.013889 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:57.013941 29933 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0414 05:13:57.014010 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0414 05:13:57.014042 29933 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0414 05:13:57.013543 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0414 05:13:57.015641 29933 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0414 05:13:57.015824 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.015931 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0414 05:13:57.016045 29933 api.go:53] PageSize not implemented I0414 05:13:57.015993 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0414 05:13:57.016171 29933 api.go:53] PageSize not implemented I0414 05:13:57.016202 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0414 05:13:57.016172 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0414 05:13:57.016301 29933 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.016374 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0414 05:13:57.016413 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0414 05:13:57.016440 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:57.016470 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0414 05:13:57.016515 29933 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0414 05:13:57.016556 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0414 05:13:57.016565 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0414 05:13:57.016613 29933 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0414 05:13:57.016655 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0414 05:13:57.016676 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0414 05:13:57.016707 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0414 05:13:57.016647 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.016841 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.016997 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0414 05:13:57.017046 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0414 05:13:57.017087 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0414 05:13:57.017227 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0414 05:13:57.017469 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0414 05:13:57.017609 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0414 05:13:57.017756 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.017887 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0414 05:13:57.017962 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0414 05:13:57.018089 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0414 05:13:57.018158 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.018233 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0414 05:13:57.018274 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0414 05:13:57.018366 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0414 05:13:57.018414 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0414 05:13:57.018595 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.018732 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0414 05:13:57.018863 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.019095 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0414 05:13:57.019301 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.019486 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.019592 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.019738 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.019849 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0414 05:13:57.019920 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:13:57.020053 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0414 05:13:57.020161 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0414 05:13:57.020295 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.020388 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0414 05:13:57.020432 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0414 05:13:57.020556 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0414 05:13:57.020597 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:57.020657 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:57.020703 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.020844 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.020970 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.021105 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.021237 29933 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0414 05:13:57.021287 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I0414 05:13:57.021419 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0414 05:13:57.021541 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0414 05:13:57.021678 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.021724 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0414 05:13:57.021757 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0414 05:13:57.021787 29933 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0414 05:13:57.021841 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.021907 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0414 05:13:57.021982 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:57.022054 29933 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0414 05:13:57.022122 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:57.022188 29933 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0414 05:13:57.024416 29933 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0414 05:13:57.024530 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.024589 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.024699 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.024757 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0414 05:13:57.024817 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.025034 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0414 05:13:57.025148 29933 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0414 05:13:57.025242 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.025295 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.025376 29933 dns.go:238] Pre-creating DNS records I0414 05:13:57.025404 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.031772 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:57.034833 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.034925 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.034994 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.072654 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.173397 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.175897 29933 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0414 05:13:57.176299 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.176493 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.176682 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.176788 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.176755 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:57.176926 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.177136 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.177340 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:57.177400 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:57.177490 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.179176 29933 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0414 05:13:57.179312 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0414 05:13:57.179486 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0414 05:13:57.179552 29933 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0414 05:13:57.179478 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.179638 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0414 05:13:57.179655 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.179828 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0414 05:13:57.180073 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.180161 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0414 05:13:57.180327 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:13:57.180383 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.180474 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0414 05:13:57.180653 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.180710 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.180853 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0414 05:13:57.180903 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0414 05:13:57.181117 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:13:57.181192 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:57.181283 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.181397 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.181500 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.181586 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.181687 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:57.181718 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.181808 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:57.181838 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.181921 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:57.184348 29933 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0414 05:13:57.184525 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0414 05:13:57.184596 29933 api.go:53] PageSize not implemented I0414 05:13:57.184621 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0414 05:13:57.184642 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0414 05:13:57.184748 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.184804 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0414 05:13:57.184834 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.184859 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0414 05:13:57.184970 29933 api.go:53] PageSize not implemented I0414 05:13:57.184991 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0414 05:13:57.185002 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0414 05:13:57.185100 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0414 05:13:57.185104 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0414 05:13:57.185150 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0414 05:13:57.185213 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.185343 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0414 05:13:57.185405 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0414 05:13:57.185492 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.186110 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.186225 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0414 05:13:57.186341 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.186526 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:13:57.189216 29933 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0414 05:13:57.189344 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.189470 29933 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0414 05:13:57.189344 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0414 05:13:57.189488 29933 api.go:53] PageSize not implemented I0414 05:13:57.189549 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.189521 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } W0414 05:13:57.190457 29933 api.go:53] PageSize not implemented I0414 05:13:57.190499 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0414 05:13:57.190651 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0414 05:13:57.190920 29933 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0414 05:13:57.196450 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:57.196502 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:57.196528 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:57.196551 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:57.196576 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:57.196603 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:57.196629 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:57.196653 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:57.196679 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:13:57.196706 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:57.196733 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:57.196760 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0414 05:13:57.196790 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:57.196816 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:57.196839 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.196867 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:57.196891 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:57.196917 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0414 05:13:57.196942 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:57.196968 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:57.196992 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:57.197015 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:13:57.197040 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:13:57.197068 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.197096 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:57.197120 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:57.197146 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0414 05:13:57.197172 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0414 05:13:57.197198 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:57.197219 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:57.197242 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:57.197269 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:57.197326 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:57.197356 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:57.198520 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:57.198539 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:57.198566 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198609 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0414 05:13:57.198664 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198705 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198738 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198774 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198802 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198839 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0414 05:13:57.198886 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.198934 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:57.198957 29933 natgateway.go:117] DescribeNatGateways: { } I0414 05:13:57.198984 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0414 05:13:57.199014 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0414 05:13:57.199049 29933 api.go:53] PageSize not implemented I0414 05:13:57.199063 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0414 05:13:57.199106 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:57.199125 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:57.199138 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:57.199156 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:57.199169 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:57.199205 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.199224 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.199260 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:57.199289 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.199322 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:57.199355 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:57.199391 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2","rtb-3"] } I0414 05:13:57.199423 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0414 05:13:57.199445 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0414 05:13:57.199900 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0414 05:13:57.199959 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0414 05:13:57.199994 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0414 05:13:57.199974 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0414 05:13:57.200050 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0414 05:13:57.200147 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0414 05:13:57.200029 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } autoscaling-group:bastion.privatekopeio.example.com ok iam-instance-profile:bastions.privatekopeio.example.com ok I0414 05:13:57.200083 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I0414 05:13:57.200277 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0414 05:13:57.200318 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0414 05:13:57.200103 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } iam-instance-profile:masters.privatekopeio.example.com ok W0414 05:13:57.200396 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-group:nodes.privatekopeio.example.com ok iam-instance-profile:nodes.privatekopeio.example.com ok I0414 05:13:57.200428 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0414 05:13:57.200493 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:13:57.200576 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0414 05:13:57.200736 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.200765 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-1 ok I0414 05:13:57.200844 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:13:57.200931 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0414 05:13:57.200998 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0414 05:13:57.201081 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0414 05:13:57.201143 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:13:57.201224 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0414 05:13:57.201323 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.201357 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0414 05:13:57.201395 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-3 ok I0414 05:13:57.201595 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } I0414 05:13:57.201597 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0414 05:13:57.201669 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.privatekopeio.example.com subnet:subnet-2 ok I0414 05:13:57.201656 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:57.201686 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0414 05:13:57.201726 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0414 05:13:57.201758 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0414 05:13:57.201800 29933 iamrole.go:182] ListAttachedRolePolicies: masters.privatekopeio.example.com I0414 05:13:57.201813 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0414 05:13:57.201725 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0414 05:13:57.201863 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.201887 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0414 05:13:57.201835 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0414 05:13:57.201937 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0414 05:13:57.201959 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0414 05:13:57.202029 29933 iamrole.go:182] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0414 05:13:57.202039 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0414 05:13:57.202039 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:57.202097 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.202063 29933 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0414 05:13:57.202160 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0414 05:13:57.202256 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.202277 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:57.202322 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0414 05:13:57.202422 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:13:57.202489 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0414 05:13:57.202547 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0414 05:13:57.202625 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:13:57.202707 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:13:57.203001 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.34s) === RUN TestLifecycleIPv6 I0414 05:13:57.203518 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:57.203567 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:57.203605 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:57.203654 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:57.203699 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:57.203731 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:57.203765 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:57.203801 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:57.203829 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:57.203889 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0414 05:13:57.230223 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:57.233438 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.233533 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.271708 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.403401 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.404660 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0414 05:13:57.404786 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.404914 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.404981 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:57.404980 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0414 05:13:57.405104 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.405152 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:57.405195 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.405225 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.405270 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.405124 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0414 05:13:57.405403 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0414 05:13:57.405412 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:57.405529 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:57.405565 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:13:57.405593 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:13:57.405524 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0414 05:13:57.405613 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:57.405639 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.405672 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.405705 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.405758 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0414 05:13:57.405799 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:57.405862 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] } I0414 05:13:57.405942 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:57.405993 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.407227 29933 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0414 05:13:57.407414 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.407552 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0414 05:13:57.407542 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.407606 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0414 05:13:57.407640 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0414 05:13:57.407658 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:57.407727 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0414 05:13:57.407715 29933 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0414 05:13:57.407722 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.407770 29933 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0414 05:13:57.407784 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0414 05:13:57.407793 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0414 05:13:57.407861 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0414 05:13:57.407888 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.407982 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0414 05:13:57.408087 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0414 05:13:57.408160 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:57.408194 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0414 05:13:57.408329 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0414 05:13:57.408402 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:57.408430 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.408454 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.408556 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:13:57.408596 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0414 05:13:57.408817 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0414 05:13:57.408946 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:57.408794 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0414 05:13:57.409023 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:57.409066 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:57.409120 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:57.409153 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:57.409177 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.409206 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0414 05:13:57.409251 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:57.409278 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.409297 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:57.409322 29933 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] } I0414 05:13:57.412205 29933 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0414 05:13:57.412367 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.412428 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.412747 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.412973 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.413187 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.413336 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0414 05:13:57.413557 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0414 05:13:57.413702 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.413792 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.413873 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:57.413922 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.413986 29933 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.414059 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.414196 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.414245 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:13:57.414329 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.414442 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0414 05:13:57.414586 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0414 05:13:57.414678 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.414809 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.415016 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.415175 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.415335 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0414 05:13:57.415455 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.415625 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0414 05:13:57.415700 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.415826 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.415895 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0414 05:13:57.416156 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.416390 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.416633 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:57.416751 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.416829 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0414 05:13:57.416885 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.416983 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.417065 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0414 05:13:57.417147 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0414 05:13:57.417238 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:57.418843 29933 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0414 05:13:57.418927 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.419026 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:57.419055 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0414 05:13:57.419078 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.419119 29933 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.419071 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:57.419180 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:57.419186 29933 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0414 05:13:57.419244 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0414 05:13:57.419273 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.419297 29933 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0414 05:13:57.419313 29933 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0414 05:13:57.419434 29933 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0414 05:13:57.419519 29933 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0414 05:13:57.419619 29933 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0414 05:13:57.419683 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.419714 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.419759 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.419924 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0414 05:13:57.419940 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.420030 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.420155 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:13:57.420221 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.420253 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.420312 29933 dns.go:238] Pre-creating DNS records I0414 05:13:57.420339 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.427022 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:57.432810 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.432911 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.470401 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.598464 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.600226 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0414 05:13:57.600716 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.600885 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.600917 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:57.600993 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.603100 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.603269 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.603458 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I0414 05:13:57.603681 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:57.603773 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:57.603844 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.605323 29933 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0414 05:13:57.606077 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.606234 29933 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0414 05:13:57.606195 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0414 05:13:57.606291 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0414 05:13:57.606247 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.606304 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0414 05:13:57.606391 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0414 05:13:57.606502 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.606544 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.606609 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:57.606652 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.606719 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0414 05:13:57.606799 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.606825 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0414 05:13:57.606905 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:57.608605 29933 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0414 05:13:57.608671 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.608773 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.608840 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.609090 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.609176 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.609424 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.609558 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.609695 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:57.609750 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.609828 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:13:57.609870 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0414 05:13:57.609916 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.610063 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0414 05:13:57.610139 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.613237 29933 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0414 05:13:57.613343 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.613373 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:57.613457 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:57.613444 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.613476 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0414 05:13:57.613549 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0414 05:13:57.613506 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:13:57.613593 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0414 05:13:57.614308 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0414 05:13:57.614349 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0414 05:13:57.614485 29933 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0414 05:13:57.614572 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:57.614580 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.614675 29933 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0414 05:13:57.614702 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:57.614725 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0414 05:13:57.614749 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:57.614828 29933 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0414 05:13:57.614849 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:57.614870 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0414 05:13:57.614954 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:13:57.618363 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:57.618429 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:57.618470 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:57.618504 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:57.618544 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0414 05:13:57.618583 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:57.618618 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:57.618653 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:57.618687 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:57.618724 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:13:57.618760 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:57.618802 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:57.618847 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0414 05:13:57.618885 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:57.618919 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:57.618953 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.618990 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:57.619024 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:57.619061 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:57.619098 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:57.619137 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:57.619174 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.619213 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:57.619249 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:57.619284 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:57.619318 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:57.619356 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:57.619395 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:57.619432 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:57.621058 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:57.621090 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:57.621133 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621191 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0414 05:13:57.621257 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621318 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621372 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621428 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621472 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621521 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0414 05:13:57.621577 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621627 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:57.621654 29933 natgateway.go:117] DescribeNatGateways: { } I0414 05:13:57.621696 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0414 05:13:57.621740 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0414 05:13:57.621795 29933 api.go:53] PageSize not implemented I0414 05:13:57.621815 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:57.621842 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:57.621856 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0414 05:13:57.621896 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:57.621920 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:57.621935 29933 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0414 05:13:57.621993 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.622021 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.622064 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:57.622097 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.622139 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:57.622188 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:57.622245 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0414 05:13:57.622285 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:13:57.622318 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0414 05:13:57.622924 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0414 05:13:57.622939 29933 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } I0414 05:13:57.622991 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0414 05:13:57.622994 29933 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0414 05:13:57.623027 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } autoscaling-group:nodes.minimal-ipv6.example.com ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok nat-gateway:nat-1 ok I0414 05:13:57.623071 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-config:lt-2 ok iam-instance-profile:masters.minimal-ipv6.example.com ok I0414 05:13:57.623114 29933 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0414 05:13:57.623108 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0414 05:13:57.623194 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.623206 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:57.623176 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } route-table:rtb-1 ok I0414 05:13:57.623227 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:13:57.623260 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0414 05:13:57.623257 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I0414 05:13:57.623282 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0414 05:13:57.623317 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0414 05:13:57.623359 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:57.623170 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal-ipv6.example.com ok I0414 05:13:57.623386 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0414 05:13:57.623545 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.623556 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0414 05:13:57.623580 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.623652 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:13:57.623700 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0414 05:13:57.623806 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0414 05:13:57.623812 29933 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0414 05:13:57.623863 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0414 05:13:57.623902 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:57.623936 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0414 05:13:57.623984 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0414 05:13:57.624008 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0414 05:13:57.624038 29933 iamrole.go:182] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0414 05:13:57.624046 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0414 05:13:57.624061 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0414 05:13:57.624111 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0414 05:13:57.624159 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:13:57.624209 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:13:57.624368 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.42s) === RUN TestLifecycleSharedVPC I0414 05:13:57.624711 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:57.624738 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:57.624764 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:57.624790 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:57.624815 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:57.624832 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:57.624850 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:57.624869 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:57.624884 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:57.624916 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0414 05:13:57.638263 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:57.639814 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.639866 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.678751 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.785692 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.787233 29933 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0414 05:13:57.787623 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0414 05:13:57.787664 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:57.787738 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0414 05:13:57.787745 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.787835 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:57.787916 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:13:57.787950 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:13:57.788001 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.788073 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0414 05:13:57.788098 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.788227 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0414 05:13:57.788308 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0414 05:13:57.788313 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0414 05:13:57.788365 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0414 05:13:57.788453 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.788576 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:57.788663 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:57.788695 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:57.788759 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:57.791381 29933 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0414 05:13:57.791502 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0414 05:13:57.791524 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.791736 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0414 05:13:57.791791 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0414 05:13:57.791777 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:57.792086 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.792134 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0414 05:13:57.792252 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:57.792442 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0414 05:13:57.792575 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.792529 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0414 05:13:57.792718 29933 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0414 05:13:57.792649 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0414 05:13:57.792807 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0414 05:13:57.793060 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0414 05:13:57.793431 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I0414 05:13:57.795200 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:57.795282 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.795361 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.795492 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:57.795560 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:13:57.795684 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:57.795776 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.796115 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0414 05:13:57.798811 29933 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0414 05:13:57.798948 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0414 05:13:57.799121 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:13:57.799298 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0414 05:13:57.799404 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0414 05:13:57.799483 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:13:57.799598 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" }] }] } I0414 05:13:57.799715 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I0414 05:13:57.799791 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.799870 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0414 05:13:57.799901 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0414 05:13:57.799984 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0414 05:13:57.800055 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.800146 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:57.800180 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0414 05:13:57.800214 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.800273 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.800361 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0414 05:13:57.800460 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I0414 05:13:57.800574 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.800731 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0414 05:13:57.800874 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" }] }] } I0414 05:13:57.801142 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.801250 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.801328 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0414 05:13:57.801544 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0414 05:13:57.803661 29933 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0414 05:13:57.804220 29933 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0414 05:13:57.804283 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.804319 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.804386 29933 dns.go:238] Pre-creating DNS records I0414 05:13:57.804417 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.809879 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:57.812999 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.813095 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.850670 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.934635 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.936500 29933 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0414 05:13:57.936856 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.936948 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:57.937155 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.937410 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:57.937566 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:13:57.937623 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:13:57.937715 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.937812 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:57.939669 29933 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0414 05:13:57.939819 29933 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0414 05:13:57.939897 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.940043 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0414 05:13:57.940129 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:57.940186 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.940393 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0414 05:13:57.940659 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:13:57.940822 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0414 05:13:57.941044 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.941128 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.944685 29933 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0414 05:13:57.944923 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.945071 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.945287 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.945502 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.945678 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.945890 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.946012 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:57.947628 29933 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0414 05:13:57.947879 29933 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0414 05:13:57.950603 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:57.950690 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:57.950731 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:57.950764 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:57.950800 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:57.950834 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:57.950870 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:13:57.950911 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:57.950948 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:57.950992 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:57.951027 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:57.951060 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:57.951094 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:57.951127 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:57.951164 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:57.951199 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:57.951237 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:57.951270 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:57.951303 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:57.951336 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:57.951373 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:57.951410 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:57.952796 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:57.952826 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:57.952866 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.952917 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0414 05:13:57.952975 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953037 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953081 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953124 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953166 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953205 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0414 05:13:57.953254 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953314 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0414 05:13:57.953352 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0414 05:13:57.953396 29933 api.go:53] PageSize not implemented I0414 05:13:57.953418 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:57.953446 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:57.953466 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:57.953490 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:57.953509 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:57.953559 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:57.953591 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:57.953638 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:57.953677 29933 iamrole.go:91] ListRoles: { } I0414 05:13:57.953719 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:57.953759 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:57.953789 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:57.954240 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:nodes.sharedvpc.example.com ok I0414 05:13:57.954305 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0414 05:13:57.954327 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:57.954422 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0414 05:13:57.954484 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0414 05:13:57.954561 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0414 05:13:57.954455 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok W0414 05:13:57.954668 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.954428 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedvpc.example.com. ok I0414 05:13:57.954694 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:57.954802 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0414 05:13:57.954879 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0414 05:13:57.954973 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:57.955051 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0414 05:13:57.955288 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:57.955311 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:13:57.955429 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0414 05:13:57.955489 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:57.955492 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0414 05:13:57.955623 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0414 05:13:57.955643 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0414 05:13:57.955695 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0414 05:13:57.955740 29933 iamrole.go:182] ListAttachedRolePolicies: masters.sharedvpc.example.com I0414 05:13:57.955763 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0414 05:13:57.955798 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0414 05:13:57.955897 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:13:57.956177 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.33s) === RUN TestLifecycleComplex I0414 05:13:57.956774 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:57.956819 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:57.956857 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:57.956905 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:57.956946 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:57.956973 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:57.957002 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:57.957032 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:57.957057 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:57.957113 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0414 05:13:57.995632 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.000460 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.000563 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.040910 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.120536 29933 iamrole.go:91] ListRoles: { } I0414 05:13:58.123038 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0414 05:13:58.123251 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:58.123449 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } W0414 05:13:58.123586 29933 api.go:53] PageSize not implemented I0414 05:13:58.123563 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" } ] } I0414 05:13:58.123628 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.123662 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0414 05:13:58.123758 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:58.123894 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.123926 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0414 05:13:58.124018 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0414 05:13:58.124174 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0414 05:13:58.124027 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:58.124335 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:58.124430 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0414 05:13:58.124564 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:58.124661 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:58.124703 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.124935 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:58.125023 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:58.125078 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:58.124990 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0414 05:13:58.125128 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.125211 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:58.125252 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:13:58.125272 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:13:58.125288 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:58.125313 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.126398 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:58.126543 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:58.126620 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.126744 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.126783 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0414 05:13:58.126868 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0414 05:13:58.126919 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:58.126819 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0414 05:13:58.126798 29933 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0414 05:13:58.126934 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:58.127002 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0414 05:13:58.127008 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:58.127063 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0414 05:13:58.127133 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:58.127174 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:58.127121 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0414 05:13:58.127202 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.127249 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0414 05:13:58.127344 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:58.127247 29933 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0414 05:13:58.127311 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.127515 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-1" } I0414 05:13:58.127546 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.127674 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:58.127750 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.127820 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0414 05:13:58.127911 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:58.127972 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:58.128019 29933 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0414 05:13:58.128125 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:58.128234 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:58.128340 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.128416 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.128511 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.128588 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0414 05:13:58.128572 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0414 05:13:58.128693 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.128801 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0414 05:13:58.128975 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:58.129078 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:58.129151 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0414 05:13:58.128889 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0414 05:13:58.129278 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:58.129372 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:58.129442 29933 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0414 05:13:58.129503 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:58.129572 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:58.129625 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:58.129701 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0414 05:13:58.129824 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:13:58.129874 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:58.129954 29933 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0414 05:13:58.130005 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:58.130087 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0414 05:13:58.130146 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:58.130201 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:58.130296 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:58.130429 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.130509 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.130639 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:58.130768 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:58.130714 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.130899 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:58.130905 29933 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], Type: "network" } I0414 05:13:58.131029 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.131100 29933 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0414 05:13:58.131232 29933 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0414 05:13:58.131226 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0414 05:13:58.131332 29933 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.131366 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.131546 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0414 05:13:58.131640 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.131743 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0414 05:13:58.131800 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0414 05:13:58.131933 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0414 05:13:58.132177 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0414 05:13:58.132340 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0414 05:13:58.132510 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.132661 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.132776 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0414 05:13:58.132815 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0414 05:13:58.132970 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0414 05:13:58.133112 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.133288 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0414 05:13:58.133457 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0414 05:13:58.133621 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0414 05:13:58.133783 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.133893 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.133982 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I0414 05:13:58.134072 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0414 05:13:58.134124 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0414 05:13:58.134245 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0414 05:13:58.134344 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.134504 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.134612 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.134695 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0414 05:13:58.134790 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0414 05:13:58.134857 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.135137 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0414 05:13:58.135406 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I0414 05:13:58.135672 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0414 05:13:58.135842 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0414 05:13:58.135937 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0414 05:13:58.136105 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0414 05:13:58.136271 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0414 05:13:58.136444 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0414 05:13:58.136716 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I0414 05:13:58.137004 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0414 05:13:58.137309 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0414 05:13:58.138008 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:58.138125 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.138334 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.138475 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0414 05:13:58.138528 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.138560 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:58.138617 29933 dns.go:238] Pre-creating DNS records I0414 05:13:58.138642 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:58.146120 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.149903 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.150007 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.187520 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.268829 29933 iamrole.go:91] ListRoles: { } I0414 05:13:58.270281 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0414 05:13:58.270424 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.270551 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:58.270776 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:58.270861 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:58.270988 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0414 05:13:58.271088 29933 api.go:53] PageSize not implemented I0414 05:13:58.271085 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.271106 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:58.271213 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:58.271234 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:58.271268 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.271292 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0414 05:13:58.271376 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.274819 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:58.275129 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.275524 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:58.275157 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:58.275733 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:58.275778 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.275475 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0414 05:13:58.277324 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:58.277366 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.277336 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:58.277425 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.277451 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.277510 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.277596 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:58.277622 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.277696 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.277759 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:58.277852 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:58.277884 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.277963 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:58.277986 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.278034 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:58.278131 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:58.278176 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:58.278341 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:58.278385 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.282030 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:58.282258 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.282347 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0414 05:13:58.282387 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0414 05:13:58.282445 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:58.282467 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.282469 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.282562 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.282607 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.282636 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.282657 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.282655 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.282677 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.282757 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.282834 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.282942 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.283412 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.283528 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.283624 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.284856 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:58.285015 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.285098 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:58.285123 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:58.285143 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.285317 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0414 05:13:58.292211 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.295591 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.307112 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.311173 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.311266 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.348205 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.438640 29933 iamrole.go:91] ListRoles: { } I0414 05:13:58.440877 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0414 05:13:58.441176 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0414 05:13:58.441317 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0414 05:13:58.441575 29933 api.go:53] PageSize not implemented I0414 05:13:58.441424 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.441682 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:58.441902 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:58.442068 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:58.442226 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:58.442391 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.442655 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.442833 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:58.442883 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:58.442976 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.444307 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:58.444541 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:58.444606 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:58.444724 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:58.444729 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.444460 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.444855 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:58.444926 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.444915 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:58.444990 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0414 05:13:58.445154 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:58.445194 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.445340 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:58.445380 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.445472 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.445601 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:58.445761 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:58.445803 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.445873 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.446001 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.446094 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.446211 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.446272 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:58.446396 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:58.446511 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:58.446577 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:58.449540 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:58.449703 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.449811 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.449988 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.449968 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:58.450091 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:58.450124 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.450197 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.450248 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.450301 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.450269 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.450356 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.450402 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.450433 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.450648 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0414 05:13:58.450705 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.450759 29933 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0414 05:13:58.450811 29933 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0414 05:13:58.450863 29933 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0414 05:13:58.451020 29933 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0414 05:13:58.451137 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.451198 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.451155 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.451318 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.451436 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.451557 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.451724 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.454090 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:58.454184 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.454289 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:58.454311 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:58.454335 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.454416 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0414 05:13:58.454465 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.454498 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:58.454551 29933 dns.go:238] Pre-creating DNS records I0414 05:13:58.463671 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.465775 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.465837 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.505686 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.572242 29933 iamrole.go:91] ListRoles: { } I0414 05:13:58.573690 29933 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0414 05:13:58.573843 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.574892 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0414 05:13:58.574930 29933 api.go:53] PageSize not implemented I0414 05:13:58.574985 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.575101 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:58.575388 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:58.574004 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:58.575585 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:58.575786 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.576098 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.576255 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:58.576306 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:58.576366 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.579981 29933 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0414 05:13:58.580139 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:58.580255 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:58.580229 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:58.580301 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.580359 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.580444 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:58.580383 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:58.580436 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:58.580559 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.580605 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:58.580703 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.580863 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:58.580908 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0414 05:13:58.581048 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:58.581098 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:58.581410 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:58.581490 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.581698 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.581856 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.582016 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:58.582076 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.582240 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.582392 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.582557 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.582777 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:58.585231 29933 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0414 05:13:58.585509 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.585637 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:58.585719 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:58.585661 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.585737 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.585779 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.585815 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.585853 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.585885 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.585836 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.585915 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.586045 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.586314 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.586550 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.586707 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.586816 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.586895 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.586956 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.589122 29933 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0414 05:13:58.589224 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.589345 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:58.589382 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:58.589419 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.589526 29933 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0414 05:13:58.593779 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.597600 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.609167 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.612832 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.612926 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.650931 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.758010 29933 iamrole.go:91] ListRoles: { } I0414 05:13:58.760259 29933 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0414 05:13:58.760584 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0414 05:13:58.760706 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:58.760722 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0414 05:13:58.760849 29933 api.go:53] PageSize not implemented I0414 05:13:58.760787 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.760946 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.761085 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.761195 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:58.761233 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:58.761265 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:58.761372 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:58.761464 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:58.761552 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0414 05:13:58.761691 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.763346 29933 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0414 05:13:58.763499 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:58.763499 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.763676 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:58.763519 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:58.763725 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.763662 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.763808 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:58.763874 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.763879 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:58.764088 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:58.764150 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.764299 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:58.764348 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.764478 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.764613 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:58.764660 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.764747 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.764865 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.764945 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.765070 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:58.765109 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:58.765217 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:58.765343 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.765418 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:58.765493 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:58.768852 29933 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0414 05:13:58.768993 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.769049 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:58.769097 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:58.769108 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.769135 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.769158 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.769139 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.769192 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.769285 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.769294 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.769376 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.769392 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.769504 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.769629 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.769990 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.770094 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.770325 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.770694 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.772046 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.772139 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.776446 29933 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0414 05:13:58.776629 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.776735 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:58.776762 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:58.776791 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.776878 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0414 05:13:58.776929 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.776964 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:58.777019 29933 dns.go:238] Pre-creating DNS records I0414 05:13:58.784597 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.789076 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.789179 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.825550 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.915583 29933 iamrole.go:91] ListRoles: { } I0414 05:13:58.917187 29933 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0414 05:13:58.917354 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:58.917449 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0414 05:13:58.918091 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0414 05:13:58.918861 29933 api.go:53] PageSize not implemented I0414 05:13:58.917511 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:58.921572 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.921741 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.921837 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.921919 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:58.921941 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:58.921981 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.922011 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:58.922082 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.922269 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:58.923550 29933 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0414 05:13:58.923701 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:58.923675 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.923776 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:58.923815 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.923853 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:58.923975 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.923947 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.924052 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:58.924132 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.924113 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.924285 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:58.924329 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.924471 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:58.924532 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0414 05:13:58.924674 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:58.924719 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.924814 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.924902 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:58.925037 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:58.925128 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:58.925298 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:58.925349 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:58.925478 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:58.925654 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:58.925715 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:58.929068 29933 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0414 05:13:58.929171 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.929272 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.929280 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:58.929325 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:58.929336 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.929365 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.929386 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.929409 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:58.929430 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:58.929377 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.929452 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:58.929512 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.929691 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.929902 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:58.930282 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.930386 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:58.930600 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.930726 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.932174 29933 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0414 05:13:58.932299 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:58.932356 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:58.932370 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:58.932385 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:58.932431 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0414 05:13:58.935457 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:58.937656 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.937708 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.942877 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:58.944917 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.944972 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.945006 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:58.982458 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.084383 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.086668 29933 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0414 05:13:59.086872 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.087067 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:59.087073 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.087170 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:59.087277 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0414 05:13:59.087416 29933 api.go:53] PageSize not implemented I0414 05:13:59.087346 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.087536 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.087690 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:59.087804 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.087915 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:59.087947 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:59.087986 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0414 05:13:59.088113 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.089052 29933 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0414 05:13:59.089185 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0414 05:13:59.089325 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:59.089388 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.089361 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:59.089462 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:59.089322 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:59.089419 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:59.089495 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.089550 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:59.089612 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.089702 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.089758 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.089834 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.089907 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:59.090007 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.090055 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.090155 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:59.090312 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.090371 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.090510 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.090691 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:59.090738 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:59.090777 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.090853 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0414 05:13:59.090982 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:59.092432 29933 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0414 05:13:59.092559 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.092630 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:59.092699 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:59.092690 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.092717 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.092772 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.092790 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.092859 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.092876 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.092900 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.092923 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.092955 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.093079 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.094275 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.094468 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.094556 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.094722 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.095063 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.098361 29933 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0414 05:13:59.098424 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:59.098480 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:59.098495 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:59.098508 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.098762 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0414 05:13:59.098830 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.098868 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:59.098931 29933 dns.go:238] Pre-creating DNS records I0414 05:13:59.107555 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:59.111382 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.111483 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.111548 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.149505 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.214212 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.217595 29933 executor.go:111] Tasks: 0 done / 117 total; 49 can run W0414 05:13:59.217784 29933 api.go:53] PageSize not implemented I0414 05:13:59.217825 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.217904 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:59.218191 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.218402 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0414 05:13:59.218598 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.218774 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0414 05:13:59.218878 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.218988 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:59.219020 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:59.219084 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.219123 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:59.219220 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:59.219312 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.220149 29933 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0414 05:13:59.220275 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:59.220333 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:59.220319 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.220411 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.220522 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:59.220594 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.220512 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.220652 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:59.220688 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.220697 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.220868 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.220986 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.221126 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.221315 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:59.221357 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.221499 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:59.221539 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.221666 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:59.221822 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:59.221864 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:59.221905 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:59.222036 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.222136 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:59.222291 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.222394 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.225655 29933 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0414 05:13:59.225895 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.225989 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.226065 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:59.226123 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:59.226138 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.226137 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.226182 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.226261 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.226299 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.226332 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.226361 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.226498 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.226685 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.226905 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.227012 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.227149 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.227463 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.227622 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.229019 29933 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0414 05:13:59.229133 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:59.229229 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:59.229256 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:59.229286 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.229376 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0414 05:13:59.232347 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.235619 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.235706 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.235763 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.244955 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:59.248640 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.248732 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.248793 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.248843 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.285517 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.367613 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.369828 29933 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0414 05:13:59.370065 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:59.370229 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.370401 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.370460 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.370532 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0414 05:13:59.370584 29933 api.go:53] PageSize not implemented I0414 05:13:59.370611 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.370914 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:59.371077 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.371219 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:59.371255 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:59.371318 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.371357 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:59.371457 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0414 05:13:59.373061 29933 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0414 05:13:59.373246 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:59.373434 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:59.373455 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.373392 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:59.373532 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.373608 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.373768 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:59.373667 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:59.373806 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.373841 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:59.373914 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:59.374117 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.374171 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:59.374335 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.374452 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.374639 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.374740 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:59.374816 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.374907 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.374996 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.375079 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.375215 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:59.375264 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.375457 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:59.375516 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.376561 29933 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0414 05:13:59.376759 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:59.376828 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:59.376845 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.376789 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.376892 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.376930 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.376941 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.376972 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.377060 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.377127 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.377109 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.377344 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.377699 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.377826 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.378091 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.378430 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.378559 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.379238 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.384824 29933 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0414 05:13:59.384926 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:59.385018 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:59.385046 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:59.385071 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.385194 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0414 05:13:59.385257 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.385294 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:59.385357 29933 dns.go:238] Pre-creating DNS records I0414 05:13:59.393667 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:59.397428 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.397527 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.397592 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.397645 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.435113 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.511825 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.513212 29933 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0414 05:13:59.513424 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0414 05:13:59.513604 29933 api.go:53] PageSize not implemented I0414 05:13:59.513588 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.513658 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:59.513679 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:59.513717 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0414 05:13:59.513889 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.513970 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.514026 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.514113 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0414 05:13:59.514215 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.514368 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.514497 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0414 05:13:59.514595 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0414 05:13:59.516321 29933 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0414 05:13:59.516497 29933 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:59.516464 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0414 05:13:59.516464 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0414 05:13:59.516597 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.516726 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:59.516652 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.516713 29933 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0414 05:13:59.516813 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.516811 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.516975 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.517076 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:13:59.517119 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.517204 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.517345 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.517533 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.517680 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:59.517721 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0414 05:13:59.517837 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.517872 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.517967 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.518069 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.518166 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0414 05:13:59.518291 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:59.518350 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0414 05:13:59.518435 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.519433 29933 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0414 05:13:59.519528 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.519590 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.519603 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:59.519691 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:59.519665 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.519711 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.519765 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.519784 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.519851 29933 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.519906 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0414 05:13:59.519943 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0414 05:13:59.519976 29933 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.519939 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.520278 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.520444 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.520963 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.521111 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:13:59.521212 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.523513 29933 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0414 05:13:59.523608 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:13:59.523689 29933 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0414 05:13:59.523713 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0414 05:13:59.523736 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.523877 29933 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0414 05:13:59.524245 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:59.524310 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:59.524347 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:59.524381 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:59.524422 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:59.524459 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:59.524490 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:59.524525 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:59.524562 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:59.524603 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:59.524648 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:59.524682 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:59.524713 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:59.524749 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:59.524781 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:13:59.524818 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:59.524855 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:59.524892 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:13:59.524928 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:59.524965 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:59.524998 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:13:59.525038 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0414 05:13:59.525075 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:59.525111 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:59.525144 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:59.525183 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:59.525221 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:59.525258 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:59.527848 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:59.527877 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:59.527913 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.527972 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0414 05:13:59.528039 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528100 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528146 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528202 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528245 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528290 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0414 05:13:59.528348 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528419 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0414 05:13:59.528461 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0414 05:13:59.528511 29933 api.go:53] PageSize not implemented I0414 05:13:59.528529 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:59.528554 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:59.528568 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0414 05:13:59.528605 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:59.528628 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:59.528645 29933 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0414 05:13:59.528717 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.528744 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:59.528784 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:59.528821 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.528862 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:59.528912 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:59.528961 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0414 05:13:59.529532 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.529591 29933 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0414 05:13:59.529680 29933 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok autoscaling-group:nodes.complex.example.com ok I0414 05:13:59.529765 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0414 05:13:59.529768 29933 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0414 05:13:59.529853 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0414 05:13:59.529729 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok I0414 05:13:59.529598 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0414 05:13:59.529974 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:59.529996 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:59.529899 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0414 05:13:59.530044 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0414 05:13:59.530052 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0414 05:13:59.530124 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0414 05:13:59.530168 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:13:59.530232 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0414 05:13:59.530440 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:59.530464 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0414 05:13:59.530514 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0414 05:13:59.530568 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:13:59.530643 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0414 05:13:59.530702 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.530760 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:13:59.530799 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0414 05:13:59.530903 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:59.530932 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0414 05:13:59.530959 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:13:59.530992 29933 iamrole.go:182] ListAttachedRolePolicies: masters.complex.example.com I0414 05:13:59.531008 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0414 05:13:59.531040 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0414 05:13:59.531071 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0414 05:13:59.531111 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.complex.example.com I0414 05:13:59.531124 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0414 05:13:59.531150 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0414 05:13:59.531228 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0414 05:13:59.531321 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:13:59.531410 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:13:59.531691 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.58s) === RUN TestLifecycleExternalLB I0414 05:13:59.532196 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:59.532241 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:59.532281 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:59.532330 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:59.532370 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:59.532399 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:59.532429 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:59.532463 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:59.532489 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:59.532543 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0414 05:13:59.557220 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:59.560024 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.560118 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.598183 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.679279 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.680966 29933 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0414 05:13:59.681295 29933 api.go:53] PageSize not implemented I0414 05:13:59.681253 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0414 05:13:59.681364 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.681433 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0414 05:13:59.681566 29933 api.go:53] PageSize not implemented I0414 05:13:59.681401 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } W0414 05:13:59.681669 29933 api.go:53] PageSize not implemented I0414 05:13:59.681526 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.681733 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:59.682200 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.682281 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:59.682502 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.682578 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.682659 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0414 05:13:59.682723 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0414 05:13:59.681669 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:59.683705 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:59.683765 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.683932 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:59.684045 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:59.684088 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.681723 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0414 05:13:59.684211 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] } I0414 05:13:59.684222 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.684297 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:59.684323 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.684386 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:59.684356 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0414 05:13:59.684439 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:59.684513 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0414 05:13:59.684611 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.684577 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.684828 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:13:59.684873 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:13:59.684909 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:59.684964 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.686119 29933 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0414 05:13:59.687666 29933 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0414 05:13:59.687756 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.687847 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0414 05:13:59.687891 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0414 05:13:59.688009 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0414 05:13:59.687896 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0414 05:13:59.688095 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0414 05:13:59.688175 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0414 05:13:59.688454 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:13:59.688495 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.688589 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:59.688637 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.688876 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.688878 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:13:59.689018 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:59.689074 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:59.689116 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:13:59.689242 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0414 05:13:59.689318 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:59.689461 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:59.689531 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.689578 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:13:59.689663 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:59.689707 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.689753 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.690038 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.690174 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:59.690249 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:59.690300 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0414 05:13:59.691051 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0414 05:13:59.692303 29933 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0414 05:13:59.692495 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" }] }] } I0414 05:13:59.692750 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.692899 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.693121 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.693224 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.693398 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.693591 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.693744 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:59.693814 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.693948 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" }] }] } I0414 05:13:59.694168 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.694321 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.694460 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:13:59.694528 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0414 05:13:59.694760 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.694848 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.694957 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.695009 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:13:59.695086 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.695292 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0414 05:13:59.695560 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.695709 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.695841 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.695976 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0414 05:13:59.696103 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0414 05:13:59.697363 29933 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0414 05:13:59.697779 29933 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0414 05:13:59.697872 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.697915 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:59.697979 29933 dns.go:238] Pre-creating DNS records I0414 05:13:59.698011 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:59.703917 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:59.706815 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.706923 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.745543 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.834578 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.835576 29933 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0414 05:13:59.835768 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.835823 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0414 05:13:59.835962 29933 api.go:53] PageSize not implemented W0414 05:13:59.836043 29933 api.go:53] PageSize not implemented I0414 05:13:59.835966 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.836100 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:59.836224 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:59.836214 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.836284 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W0414 05:13:59.836384 29933 api.go:53] PageSize not implemented I0414 05:13:59.836441 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0414 05:13:59.836307 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.836444 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.836526 29933 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.836600 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0414 05:13:59.836613 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:59.836710 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:59.837687 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.838132 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:59.838196 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:59.838325 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.839808 29933 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0414 05:13:59.840083 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0414 05:13:59.840068 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.841743 29933 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0414 05:13:59.841807 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0414 05:13:59.842043 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.842099 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.842175 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.842259 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.842345 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:59.842369 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0414 05:13:59.842456 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.842492 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:59.846388 29933 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0414 05:13:59.846552 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.846662 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.846715 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.846825 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.846915 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:13:59.847172 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.847337 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.848270 29933 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0414 05:13:59.848495 29933 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0414 05:13:59.851653 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:13:59.851717 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:13:59.851754 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:13:59.851786 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:13:59.851826 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:13:59.851860 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:13:59.851891 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:13:59.851924 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:13:59.851957 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:13:59.851993 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:13:59.852031 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:13:59.852076 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:13:59.852107 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:13:59.852137 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:13:59.852175 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:13:59.852209 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:59.852245 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:13:59.852277 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:59.852315 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:13:59.852346 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:13:59.852378 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:13:59.852411 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:59.852447 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:59.852484 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:59.852521 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:13:59.854267 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:13:59.854296 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:13:59.854332 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854381 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0414 05:13:59.854437 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854495 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854541 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854600 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854643 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854684 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0414 05:13:59.854737 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854791 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0414 05:13:59.854833 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0414 05:13:59.854885 29933 api.go:53] PageSize not implemented I0414 05:13:59.854907 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:13:59.854935 29933 loadbalancers.go:34] PageSize not implemented I0414 05:13:59.854951 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:13:59.854975 29933 targetgroups.go:35] PageSize not implemented I0414 05:13:59.854990 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:13:59.855041 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.855073 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:13:59.855123 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:13:59.855160 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.855201 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:13:59.855247 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:13:59.855281 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok autoscaling-config:lt-1 ok autoscaling-group:nodes.externallb.example.com ok I0414 05:13:59.855854 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0414 05:13:59.855857 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:13:59.855956 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0414 05:13:59.856046 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0414 05:13:59.856012 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0414 05:13:59.856126 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0414 05:13:59.855839 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok I0414 05:13:59.856229 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0414 05:13:59.856569 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:59.856621 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:13:59.856714 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0414 05:13:59.856837 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0414 05:13:59.856906 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:13:59.856972 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0414 05:13:59.857015 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:13:59.857065 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0414 05:13:59.857185 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:13:59.857194 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:13:59.857250 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0414 05:13:59.857279 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.externallb.example.com I0414 05:13:59.857302 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0414 05:13:59.857322 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0414 05:13:59.857339 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:13:59.857384 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0414 05:13:59.857464 29933 iamrole.go:182] ListAttachedRolePolicies: masters.externallb.example.com I0414 05:13:59.857484 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0414 05:13:59.857519 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0414 05:13:59.857600 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:13:59.857685 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:13:59.857760 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:13:59.858029 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.33s) === RUN TestLifecycleSharedSubnet I0414 05:13:59.858542 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:59.858593 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:59.858635 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:59.858686 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:59.858728 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:59.858759 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:59.858792 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:59.858826 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:59.858852 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:59.858908 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0414 05:13:59.882790 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:13:59.885558 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.885655 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:13:59.924375 29933 zones.go:93] ListHostedZonesPages { } I0414 05:13:59.993157 29933 iamrole.go:91] ListRoles: { } I0414 05:13:59.994002 29933 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0414 05:13:59.994122 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:13:59.994263 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:59.994264 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0414 05:13:59.994305 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:13:59.994311 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0414 05:13:59.994398 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:13:59.994488 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:13:59.994537 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:13:59.994556 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:13:59.994584 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:13:59.994670 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] } I0414 05:13:59.994705 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:59.994806 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:13:59.994772 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0414 05:13:59.994845 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:13:59.994906 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0414 05:13:59.994968 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I0414 05:13:59.995018 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:13:59.995162 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:13:59.998418 29933 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0414 05:13:59.998726 29933 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0414 05:13:59.998857 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0414 05:13:59.998884 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0414 05:13:59.999202 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0414 05:13:59.999328 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0414 05:13:59.999403 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0414 05:13:59.999467 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.001239 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:00.001326 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:14:00.001474 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0414 05:14:00.001599 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.001766 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.001835 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:00.001940 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.001897 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0414 05:14:00.001998 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:14:00.002048 29933 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0414 05:14:00.002048 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0414 05:14:00.006616 29933 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0414 05:14:00.006761 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:14:00.006928 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0414 05:14:00.007206 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:14:00.007513 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.007634 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.007692 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:14:00.007996 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:14:00.008166 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:14:00.008256 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" }] }] } I0414 05:14:00.008344 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:14:00.008436 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0414 05:14:00.008527 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.008583 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" }] }] } I0414 05:14:00.008677 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" }] }] } I0414 05:14:00.008754 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:14:00.008829 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I0414 05:14:00.008900 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0414 05:14:00.008987 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.010383 29933 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0414 05:14:00.010673 29933 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0414 05:14:00.010736 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.010775 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.010843 29933 dns.go:238] Pre-creating DNS records I0414 05:14:00.010873 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.017170 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:14:00.020023 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.020120 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.057877 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.166247 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.167741 29933 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0414 05:14:00.168004 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:14:00.168133 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:14:00.168172 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:14:00.168254 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.168362 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.168473 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:00.168540 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.168741 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.170251 29933 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0414 05:14:00.170370 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0414 05:14:00.170441 29933 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0414 05:14:00.170441 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0414 05:14:00.170643 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:14:00.170765 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0414 05:14:00.170906 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.170971 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0414 05:14:00.171075 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.175233 29933 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0414 05:14:00.175549 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.175763 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.176032 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.176132 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.177963 29933 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0414 05:14:00.178184 29933 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0414 05:14:00.180922 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:14:00.180997 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:14:00.181070 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:14:00.181133 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:00.181198 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:14:00.181260 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:14:00.181326 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:14:00.181393 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:14:00.181463 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:14:00.181532 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:14:00.181588 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:14:00.181642 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:14:00.181702 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:00.181764 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:00.181826 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:14:00.181886 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:14:00.181943 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:14:00.181998 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:00.182064 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:00.182124 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:14:00.183559 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:14:00.183596 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:14:00.183652 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.183719 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0414 05:14:00.183808 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.183892 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.183967 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.184041 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.184106 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.184163 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0414 05:14:00.184232 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.184302 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0414 05:14:00.184353 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0414 05:14:00.184427 29933 api.go:53] PageSize not implemented I0414 05:14:00.184468 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:14:00.184513 29933 loadbalancers.go:34] PageSize not implemented I0414 05:14:00.184542 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:14:00.184579 29933 targetgroups.go:35] PageSize not implemented I0414 05:14:00.184608 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:14:00.184678 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.184727 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.184803 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:14:00.184862 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.184928 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:14:00.184987 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:14:00.185373 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.sharedsubnet.example.com ok I0414 05:14:00.185402 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0414 05:14:00.185523 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0414 05:14:00.185447 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0414 05:14:00.185974 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } W0414 05:14:00.186003 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-instance-profile:masters.sharedsubnet.example.com ok I0414 05:14:00.186039 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:14:00.186046 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } autoscaling-config:lt-2 ok I0414 05:14:00.186093 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0414 05:14:00.186139 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0414 05:14:00.186236 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.186319 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0414 05:14:00.186532 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.186560 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:14:00.186628 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0414 05:14:00.186708 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0414 05:14:00.186830 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0414 05:14:00.186889 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0414 05:14:00.186972 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0414 05:14:00.186992 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0414 05:14:00.187030 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0414 05:14:00.187064 29933 iamrole.go:182] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0414 05:14:00.187090 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0414 05:14:00.187141 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0414 05:14:00.187437 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.33s) === RUN TestLifecyclePrivateSharedSubnet I0414 05:14:00.187968 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:14:00.188030 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:14:00.188103 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:14:00.188187 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:14:00.188246 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:14:00.188304 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:14:00.188362 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:14:00.188422 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:14:00.188467 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:14:00.188555 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0414 05:14:00.222117 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:14:00.225077 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.225168 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.225232 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.263879 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.355219 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.357367 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0414 05:14:00.357800 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:14:00.357929 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:14:00.357973 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:14:00.357881 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0414 05:14:00.358046 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.358063 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.358115 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:00.358228 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0414 05:14:00.358208 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.358315 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.358386 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:14:00.358514 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:00.358383 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.358551 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:14:00.358649 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:00.358685 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.358611 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.358724 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.358730 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.358818 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.358909 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0414 05:14:00.358976 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.359752 29933 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0414 05:14:00.359877 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0414 05:14:00.359992 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0414 05:14:00.360091 29933 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0414 05:14:00.360062 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0414 05:14:00.360187 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0414 05:14:00.360334 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0414 05:14:00.360248 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0414 05:14:00.360375 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0414 05:14:00.360453 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.360554 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:00.360574 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0414 05:14:00.360614 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.360695 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0414 05:14:00.360722 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0414 05:14:00.360783 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0414 05:14:00.360869 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:14:00.360925 29933 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" },{ Key: "kubernetes.io/role/internal-elb", Value: "1" }] } I0414 05:14:00.361012 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:14:00.361170 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.361275 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:00.361329 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.361368 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0414 05:14:00.361476 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0414 05:14:00.361583 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0414 05:14:00.361691 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:14:00.361732 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.361815 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:14:00.361860 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.361985 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:14:00.362040 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:14:00.362092 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.362183 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:14:00.362227 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:14:00.362265 29933 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "SubnetType", Value: "Utility" },{ Key: "kubernetes.io/role/elb", Value: "1" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }] } I0414 05:14:00.362373 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:14:00.362237 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0414 05:14:00.362880 29933 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0414 05:14:00.363074 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0414 05:14:00.363205 29933 api.go:53] PageSize not implemented I0414 05:14:00.363156 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0414 05:14:00.363269 29933 api.go:53] PageSize not implemented I0414 05:14:00.363271 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.363334 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0414 05:14:00.363325 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0414 05:14:00.363411 29933 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" }] } I0414 05:14:00.363455 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0414 05:14:00.363448 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.363475 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0414 05:14:00.363509 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0414 05:14:00.363558 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.363594 29933 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0414 05:14:00.363629 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.363646 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0414 05:14:00.363570 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0414 05:14:00.363671 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0414 05:14:00.363805 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.363985 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.364279 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0414 05:14:00.364385 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.364488 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.364677 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.364804 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0414 05:14:00.364946 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" }] }] } I0414 05:14:00.365038 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.365114 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0414 05:14:00.365237 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.365319 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0414 05:14:00.365389 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:14:00.365448 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" }] }] } I0414 05:14:00.365525 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.365602 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.365677 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0414 05:14:00.365727 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.365796 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.365870 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0414 05:14:00.365945 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:14:00.366037 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0414 05:14:00.366110 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.366168 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.367704 29933 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0414 05:14:00.367851 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:14:00.368033 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:14:00.368278 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:14:00.368526 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.368650 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.368847 29933 dns.go:238] Pre-creating DNS records I0414 05:14:00.368952 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.374024 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:14:00.375651 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.375707 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.375745 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.413158 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.507720 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.511293 29933 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0414 05:14:00.511689 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.511875 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.511982 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:14:00.512043 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:00.512143 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:14:00.512211 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:14:00.512301 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.512411 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.513930 29933 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0414 05:14:00.514204 29933 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0414 05:14:00.514248 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0414 05:14:00.514170 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0414 05:14:00.514335 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0414 05:14:00.514481 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0414 05:14:00.514522 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0414 05:14:00.514714 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.514773 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:14:00.514849 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:14:00.514906 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:14:00.515077 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0414 05:14:00.515201 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0414 05:14:00.515365 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.515421 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0414 05:14:00.515515 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0414 05:14:00.515645 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:14:00.518990 29933 executor.go:111] Tasks: 70 done / 105 total; 31 can run W0414 05:14:00.519234 29933 api.go:53] PageSize not implemented I0414 05:14:00.519262 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.519357 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } W0414 05:14:00.519398 29933 api.go:53] PageSize not implemented I0414 05:14:00.519323 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.519430 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0414 05:14:00.519471 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.519531 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0414 05:14:00.519675 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.519702 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.519780 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0414 05:14:00.519844 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.520426 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.520591 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.521911 29933 executor.go:111] Tasks: 101 done / 105 total; 4 can run W0414 05:14:00.522024 29933 api.go:53] PageSize not implemented I0414 05:14:00.522045 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.522045 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:14:00.522158 29933 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0414 05:14:00.522188 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0414 05:14:00.522256 29933 api.go:53] PageSize not implemented I0414 05:14:00.522280 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.522494 29933 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0414 05:14:00.526011 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:14:00.526075 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:14:00.526112 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:14:00.526143 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:00.526179 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:14:00.526217 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:14:00.526252 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:14:00.526288 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:14:00.526330 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:14:00.526378 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0414 05:14:00.526417 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:14:00.526451 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:14:00.526485 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:14:00.526517 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:00.526553 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:00.526591 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:14:00.526627 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:14:00.526663 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:14:00.526699 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:14:00.526737 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:14:00.526775 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:14:00.526809 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:00.526846 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:00.526884 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:14:00.528501 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:14:00.528533 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:14:00.528575 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.528642 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0414 05:14:00.528723 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.528785 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.528829 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.528873 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.528918 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.528955 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0414 05:14:00.528995 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.529041 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0414 05:14:00.529077 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0414 05:14:00.529119 29933 api.go:53] PageSize not implemented I0414 05:14:00.529136 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0414 05:14:00.529192 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:14:00.529221 29933 loadbalancers.go:34] PageSize not implemented I0414 05:14:00.529240 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:14:00.529266 29933 targetgroups.go:35] PageSize not implemented I0414 05:14:00.529283 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:14:00.529337 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.529365 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.529413 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:14:00.529453 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.529498 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:14:00.529543 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:14:00.530055 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:bastion.private-shared-subnet.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok autoscaling-group:nodes.private-shared-subnet.example.com ok I0414 05:14:00.530162 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0414 05:14:00.530177 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:14:00.530248 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0414 05:14:00.530289 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0414 05:14:00.530337 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } keypair:key-1 ok autoscaling-config:lt-2 ok I0414 05:14:00.530366 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0414 05:14:00.530412 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0414 05:14:00.530291 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0414 05:14:00.530455 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.530433 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I0414 05:14:00.530261 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-subnet.example.com ok I0414 05:14:00.530473 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0414 05:14:00.530582 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0414 05:14:00.530604 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0414 05:14:00.530696 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0414 05:14:00.530817 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.530841 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0414 05:14:00.530904 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0414 05:14:00.531005 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0414 05:14:00.531137 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.531199 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0414 05:14:00.531400 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.531419 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0414 05:14:00.531503 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:14:00.531523 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0414 05:14:00.531586 29933 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0414 05:14:00.531552 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0414 05:14:00.531633 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.531603 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0414 05:14:00.531647 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0414 05:14:00.531667 29933 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0414 05:14:00.531678 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.531713 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0414 05:14:00.531770 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0414 05:14:00.531723 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0414 05:14:00.531817 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.531832 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:14:00.531787 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0414 05:14:00.531876 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0414 05:14:00.531905 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0414 05:14:00.531956 29933 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0414 05:14:00.531969 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0414 05:14:00.531998 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0414 05:14:00.532246 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.34s) === RUN TestLifecyclePrivateSharedIP I0414 05:14:00.532722 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:14:00.532769 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:14:00.532810 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:14:00.532857 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:14:00.532894 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:14:00.532925 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:14:00.532958 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:14:00.532989 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:14:00.533015 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:14:00.533066 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0414 05:14:00.561169 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:14:00.564119 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.564210 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.564278 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.602730 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.704414 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.706588 29933 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0414 05:14:00.706821 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.706988 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:14:00.707079 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:00.707066 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:00.707133 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.707105 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0414 05:14:00.707177 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.707265 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.707253 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0414 05:14:00.707393 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:14:00.707463 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:00.707460 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0414 05:14:00.707491 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:14:00.707531 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:14:00.707554 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:14:00.707525 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0414 05:14:00.707583 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.707650 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0414 05:14:00.707714 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0414 05:14:00.707844 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0414 05:14:00.710753 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0414 05:14:00.710837 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0414 05:14:00.712355 29933 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0414 05:14:00.713153 29933 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0414 05:14:00.713109 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0414 05:14:00.713261 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0414 05:14:00.713256 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0414 05:14:00.713373 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0414 05:14:00.713505 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0414 05:14:00.713550 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.713639 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0414 05:14:00.713810 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0414 05:14:00.713801 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0414 05:14:00.713999 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0414 05:14:00.713800 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.714094 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0414 05:14:00.714216 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0414 05:14:00.714296 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0414 05:14:00.714315 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.714549 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0414 05:14:00.714559 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0414 05:14:00.714682 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:14:00.714736 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:14:00.714782 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.714883 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:00.714931 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.714973 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.715057 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:00.715097 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.715133 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.715212 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:14:00.715293 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0414 05:14:00.715291 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0414 05:14:00.715520 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:14:00.715600 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.715659 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:14:00.715695 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:14:00.715737 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0414 05:14:00.715829 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:14:00.715868 29933 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0414 05:14:00.715943 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.716178 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:14:00.716312 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:14:00.716424 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:14:00.716555 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I0414 05:14:00.716715 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:14:00.716777 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:14:00.716818 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0414 05:14:00.716907 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:14:00.716958 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:14:00.716523 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0414 05:14:00.718563 29933 executor.go:111] Tasks: 72 done / 114 total; 36 can run W0414 05:14:00.718734 29933 api.go:53] PageSize not implemented I0414 05:14:00.718859 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.718914 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0414 05:14:00.719067 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.719049 29933 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0414 05:14:00.719232 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0414 05:14:00.719196 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.719334 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:14:00.719324 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0414 05:14:00.719456 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:14:00.719493 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0414 05:14:00.719443 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } W0414 05:14:00.719569 29933 api.go:53] PageSize not implemented I0414 05:14:00.719592 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0414 05:14:00.719583 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0414 05:14:00.719719 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0414 05:14:00.719784 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.719824 29933 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0414 05:14:00.719904 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.719834 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.719941 29933 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0414 05:14:00.719991 29933 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0414 05:14:00.720022 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0414 05:14:00.720104 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:14:00.720307 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0414 05:14:00.720391 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0414 05:14:00.720474 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:00.720556 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:14:00.720644 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0414 05:14:00.720786 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0414 05:14:00.720975 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0414 05:14:00.721154 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.721327 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.721469 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0414 05:14:00.721598 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0414 05:14:00.721746 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:00.721789 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:14:00.721907 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0414 05:14:00.722001 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0414 05:14:00.722135 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.722256 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.722394 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0414 05:14:00.722440 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0414 05:14:00.722483 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.722570 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.722723 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0414 05:14:00.722841 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.722969 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.723056 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0414 05:14:00.723156 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" }] }] } I0414 05:14:00.723292 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.723436 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0414 05:14:00.723562 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:00.723636 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0414 05:14:00.723681 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.723728 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.723856 29933 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0414 05:14:00.724776 29933 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0414 05:14:00.724871 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:14:00.724916 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:00.725025 29933 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:14:00.725021 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:14:00.725186 29933 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0414 05:14:00.725299 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0414 05:14:00.725353 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:14:00.725419 29933 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0414 05:14:00.725500 29933 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0414 05:14:00.725583 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:00.725746 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0414 05:14:00.725836 29933 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0414 05:14:00.725895 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.725947 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.726013 29933 dns.go:238] Pre-creating DNS records I0414 05:14:00.726054 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.732218 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0414 05:14:00.735287 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.735383 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.735465 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.773219 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.859381 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.861634 29933 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0414 05:14:00.862099 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.862213 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:00.862371 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:00.862451 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.862696 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.862917 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0414 05:14:00.863045 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0414 05:14:00.863094 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0414 05:14:00.864220 29933 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0414 05:14:00.864460 29933 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0414 05:14:00.864370 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0414 05:14:00.864634 29933 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0414 05:14:00.864743 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0414 05:14:00.864600 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.864918 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:14:00.865018 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0414 05:14:00.865274 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:14:00.865354 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0414 05:14:00.865582 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.865676 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0414 05:14:00.865836 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.865988 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0414 05:14:00.866039 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.867079 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:14:00.867222 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0414 05:14:00.867500 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0414 05:14:00.867740 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.867921 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:14:00.868074 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:00.868863 29933 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0414 05:14:00.869226 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0414 05:14:00.869403 29933 api.go:53] PageSize not implemented I0414 05:14:00.869382 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:14:00.869505 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.869588 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0414 05:14:00.869749 29933 api.go:53] PageSize not implemented I0414 05:14:00.869791 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0414 05:14:00.869771 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0414 05:14:00.869895 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.871578 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0414 05:14:00.871653 29933 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0414 05:14:00.871831 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.872064 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.872256 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.872397 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0414 05:14:00.872545 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:00.872695 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.872891 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.873066 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0414 05:14:00.873121 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:00.875413 29933 executor.go:111] Tasks: 108 done / 114 total; 5 can run W0414 05:14:00.875564 29933 api.go:53] PageSize not implemented I0414 05:14:00.875551 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0414 05:14:00.875606 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:00.875599 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.875682 29933 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0414 05:14:00.875706 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:14:00.875727 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0414 05:14:00.875792 29933 api.go:53] PageSize not implemented I0414 05:14:00.875809 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.875930 29933 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0414 05:14:00.875970 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:00.876032 29933 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0414 05:14:00.878060 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:14:00.878096 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:14:00.878119 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:14:00.878140 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:00.878161 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:14:00.878181 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:14:00.878201 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:14:00.878222 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:14:00.878247 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:14:00.878270 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0414 05:14:00.878293 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0414 05:14:00.878316 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:14:00.878336 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:14:00.878355 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:14:00.878376 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:14:00.878395 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0414 05:14:00.878416 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:00.878437 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:00.878458 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0414 05:14:00.878479 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0414 05:14:00.878500 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0414 05:14:00.878521 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:14:00.878543 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:14:00.878562 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0414 05:14:00.878584 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:14:00.878604 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:14:00.878624 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:00.878646 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:00.878668 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:14:00.879556 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:14:00.879574 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:14:00.879599 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879637 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0414 05:14:00.879685 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879722 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879746 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879773 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879797 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879822 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0414 05:14:00.879854 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879886 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:14:00.879904 29933 natgateway.go:117] DescribeNatGateways: { } I0414 05:14:00.879929 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0414 05:14:00.879951 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0414 05:14:00.879974 29933 api.go:53] PageSize not implemented I0414 05:14:00.879983 29933 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0414 05:14:00.880013 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:14:00.880029 29933 loadbalancers.go:34] PageSize not implemented I0414 05:14:00.880039 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:14:00.880053 29933 targetgroups.go:35] PageSize not implemented I0414 05:14:00.880063 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:14:00.880092 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:00.880109 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:00.880134 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:14:00.880156 29933 iamrole.go:91] ListRoles: { } I0414 05:14:00.880184 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:14:00.880213 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:14:00.880233 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0414 05:14:00.880255 29933 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0414 05:14:00.880275 29933 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0414 05:14:00.880634 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0414 05:14:00.880672 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0414 05:14:00.880701 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0414 05:14:00.880726 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0414 05:14:00.880695 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:nodes.private-shared-ip.example.com ok I0414 05:14:00.880794 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } volume:vol-2 ok I0414 05:14:00.880782 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:bastion.private-shared-ip.example.com ok I0414 05:14:00.880831 29933 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0414 05:14:00.880849 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0414 05:14:00.880953 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.880973 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok load-balancer:bastion-private-shared-ip-eepmph ok load-balancer:api-private-shared-ip-exa-ohatqj ok autoscaling-config:lt-3 ok I0414 05:14:00.880963 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } autoscaling-config:lt-2 ok I0414 05:14:00.881022 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0414 05:14:00.881054 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0414 05:14:00.880793 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok I0414 05:14:00.881079 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W0414 05:14:00.881277 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.881299 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0414 05:14:00.881344 29933 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok autoscaling-config:lt-1 ok I0414 05:14:00.881409 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:00.881460 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0414 05:14:00.881566 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.881583 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:14:00.881616 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0414 05:14:00.881669 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0414 05:14:00.881781 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0414 05:14:00.881803 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0414 05:14:00.881909 29933 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0414 05:14:00.881938 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0414 05:14:00.881829 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0414 05:14:00.881991 29933 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok W0414 05:14:00.882001 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.882040 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0414 05:14:00.882075 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } I0414 05:14:00.882118 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-ip.example.com security-group:sg-4 ok I0414 05:14:00.882139 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0414 05:14:00.882149 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:14:00.882182 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0414 05:14:00.882198 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0414 05:14:00.882226 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0414 05:14:00.882218 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0414 05:14:00.882303 29933 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0414 05:14:00.882259 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0414 05:14:00.882322 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:00.882339 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0414 05:14:00.882320 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0414 05:14:00.882375 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0414 05:14:00.882497 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0414 05:14:00.882600 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:14:00.882975 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.35s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0414 05:14:00.883592 29933 internetgateways.go:66] CreateInternetGateway: { } I0414 05:14:00.883640 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:14:00.883680 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:14:00.883729 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:14:00.883771 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:14:00.883802 29933 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:14:00.883835 29933 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:14:00.883869 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:14:00.883895 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:14:00.883973 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0414 05:14:00.909345 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0414 05:14:00.912273 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.912363 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:00.951855 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:01.054008 29933 iamrole.go:91] ListRoles: { } I0414 05:14:01.055152 29933 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0414 05:14:01.055345 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.055468 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0414 05:14:01.055551 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.055732 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:01.055789 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0414 05:14:01.055716 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.055884 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0414 05:14:01.055974 29933 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0414 05:14:01.055967 29933 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0414 05:14:01.056132 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:14:01.056102 29933 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0414 05:14:01.056186 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.056210 29933 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0414 05:14:01.056381 29933 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.056390 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:14:01.056579 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:01.056647 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:01.056713 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:01.056905 29933 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0414 05:14:01.056997 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:01.057039 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0414 05:14:01.057061 29933 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0414 05:14:01.057076 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:14:01.057102 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:14:01.057128 29933 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.057508 29933 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0414 05:14:01.057662 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0414 05:14:01.057703 29933 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0414 05:14:01.057697 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.057746 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0414 05:14:01.057811 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0414 05:14:01.057898 29933 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0414 05:14:01.057940 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:14:01.057996 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:01.058035 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:01.058060 29933 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.058105 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:14:01.058135 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.058190 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0414 05:14:01.058263 29933 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0414 05:14:01.058272 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:14:01.058447 29933 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:14:01.058441 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0414 05:14:01.058520 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.058703 29933 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0414 05:14:01.058728 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0414 05:14:01.058937 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:14:01.059114 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:14:01.059199 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:14:01.059292 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:14:01.059337 29933 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0414 05:14:01.059380 29933 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0414 05:14:01.059508 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:01.059565 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:01.059618 29933 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" } ] } I0414 05:14:01.059481 29933 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0414 05:14:01.062411 29933 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0414 05:14:01.062547 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.062741 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:01.062911 29933 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:14:01.063054 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0414 05:14:01.063308 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.063454 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.063567 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.063795 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.064027 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:01.064134 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:01.064256 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" }] }] } I0414 05:14:01.064453 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.064631 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.064812 29933 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" }] }] } I0414 05:14:01.064969 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.065145 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.065307 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.065487 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0414 05:14:01.065559 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.065718 29933 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0414 05:14:01.065763 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0414 05:14:01.065898 29933 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0414 05:14:01.066046 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.066140 29933 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0414 05:14:01.066235 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.067977 29933 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0414 05:14:01.068272 29933 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0414 05:14:01.068633 29933 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0414 05:14:01.068763 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:01.068847 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:01.068978 29933 dns.go:238] Pre-creating DNS records I0414 05:14:01.069028 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:01.076866 29933 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0414 05:14:01.080267 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.080374 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.118654 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:01.221597 29933 iamrole.go:91] ListRoles: { } I0414 05:14:01.222585 29933 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0414 05:14:01.222917 29933 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0414 05:14:01.223074 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.223177 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.223363 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:14:01.223407 29933 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:14:01.223467 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:14:01.223508 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:01.223581 29933 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0414 05:14:01.223682 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.223984 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.225262 29933 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0414 05:14:01.225403 29933 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0414 05:14:01.225414 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:14:01.225541 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.225639 29933 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0414 05:14:01.225684 29933 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:14:01.225772 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.225916 29933 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:14:01.225957 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0414 05:14:01.226122 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:01.226177 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0414 05:14:01.226317 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:01.226374 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.228355 29933 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0414 05:14:01.228491 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:01.228587 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:01.228775 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0414 05:14:01.228933 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.229122 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.229246 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.229419 29933 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0414 05:14:01.232920 29933 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0414 05:14:01.233348 29933 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0414 05:14:01.233443 29933 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0414 05:14:01.235470 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0414 05:14:01.235512 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0414 05:14:01.235532 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0414 05:14:01.235551 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0414 05:14:01.235572 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0414 05:14:01.235592 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0414 05:14:01.235612 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0414 05:14:01.235629 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0414 05:14:01.235650 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0414 05:14:01.235669 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0414 05:14:01.235695 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0414 05:14:01.235718 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0414 05:14:01.235738 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0414 05:14:01.235757 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0414 05:14:01.235778 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0414 05:14:01.235796 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:14:01.235816 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0414 05:14:01.235837 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:14:01.235858 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0414 05:14:01.235877 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0414 05:14:01.235895 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0414 05:14:01.235913 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0414 05:14:01.235935 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0414 05:14:01.235957 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:14:01.235978 29933 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0414 05:14:01.236795 29933 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0414 05:14:01.236812 29933 keypairs.go:95] DescribeKeyPairs: { } I0414 05:14:01.236835 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.236865 29933 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.236900 29933 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.236936 29933 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.236963 29933 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.236997 29933 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.237022 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.237046 29933 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.237074 29933 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.237106 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0414 05:14:01.237131 29933 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0414 05:14:01.237161 29933 api.go:53] PageSize not implemented I0414 05:14:01.237174 29933 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0414 05:14:01.237190 29933 loadbalancers.go:34] PageSize not implemented I0414 05:14:01.237201 29933 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0414 05:14:01.237215 29933 targetgroups.go:35] PageSize not implemented I0414 05:14:01.237226 29933 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0414 05:14:01.237255 29933 zones.go:93] ListHostedZonesPages { } I0414 05:14:01.237273 29933 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0414 05:14:01.237313 29933 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0414 05:14:01.237335 29933 iamrole.go:91] ListRoles: { } I0414 05:14:01.237360 29933 internetgateways.go:101] DescribeInternetGateways: { } I0414 05:14:01.237385 29933 routetable.go:53] DescribeRouteTables: { } I0414 05:14:01.237405 29933 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0414 05:14:01.237696 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0414 05:14:01.237701 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0414 05:14:01.237732 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I0414 05:14:01.237755 29933 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0414 05:14:01.237803 29933 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0414 05:14:01.237748 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0414 05:14:01.237888 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:01.237769 29933 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I0414 05:14:01.237900 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0414 05:14:01.237944 29933 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0414 05:14:01.237976 29933 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0414 05:14:01.238040 29933 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0414 05:14:01.238049 29933 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0414 05:14:01.238073 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0414 05:14:01.238106 29933 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0414 05:14:01.238140 29933 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0414 05:14:01.238171 29933 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0414 05:14:01.238201 29933 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0414 05:14:01.238239 29933 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0414 05:14:01.238275 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0414 05:14:01.238309 29933 iamrole.go:182] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0414 05:14:01.238319 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0414 05:14:01.238340 29933 iamrole.go:139] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0414 05:14:01.238360 29933 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0414 05:14:01.238385 29933 iamrole.go:182] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0414 05:14:01.238392 29933 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0414 05:14:01.238408 29933 iamrole.go:139] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0414 05:14:01.238445 29933 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0414 05:14:01.238489 29933 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0414 05:14:01.238531 29933 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0414 05:14:01.238674 29933 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.36s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 47.917s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.019s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0414 05:12:49.994093 24267 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.022s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.035s === RUN TestPodController I0414 05:12:53.232652 25832 pod.go:59] starting pod controller I0414 05:12:53.232928 25832 stoppable.go:62] shutting down controller I0414 05:12:53.232974 25832 pod.go:107] Got stop signal I0414 05:12:53.233015 25832 pod.go:65] shutting down pod controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.033s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.005s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a3d80} route53_test.go:93: Got zone list: [0xc0000a3db0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc000010090} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a3e80} route53_test.go:93: Got zone list: [0xc0000a3eb0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000100a0} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000a3f50} route53_test.go:170: Successfully added managed DNS zone: &{0xc000342660 0xc0000100b8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035a0b0} route53_test.go:93: Got zone list: [0xc00035a0e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000100d0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035a1e0} route53_test.go:93: Got zone list: [0xc00035a210] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000100f0} route53_test.go:187: Successfully added resource record set: {0xc0001a8100 0xc000010110} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035a490} route53_test.go:93: Got zone list: [0xc00035a4c0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc000010198} route53_test.go:199: Successfully added resource record set: {0xc0001a8280 0xc0000101b8} route53_test.go:118: Got 1 recordsets: [0xc00035a660] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035a790} route53_test.go:93: Got zone list: [0xc00035a7c0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc000010250} route53_test.go:221: Successfully added resource record set: {0xc0001a8400 0xc000010270} route53_test.go:228: Correctly failed to add duplicate resource record {0xc0001a8400 0xc000010270}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035ab20} route53_test.go:93: Got zone list: [0xc00035ab50] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc000010328} route53_test.go:246: Successfully removed resource set {0xc0001a8600 0xc000010348} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035add0} route53_test.go:93: Got zone list: [0xc00035ae00] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000103d0} route53_test.go:264: Successfully removed resource set {0xc0001a8780 0xc0000103f0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035b0b0} route53_test.go:93: Got zone list: [0xc00035b0e0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc000010480} commontests.go:192: Successfully added resource record set: {0xc0001a8900 0xc0000104a0} commontests.go:45: Correctly replaced resource record {0xc0001a8900 0xc0000104a0} -> {0xc0001a8a00 0xc0000104f0} commontests.go:142: Got 1 recordsets: [0xc00035b3f0] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035b5c0} route53_test.go:93: Got zone list: [0xc00035b5f0] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000105a0} commontests.go:192: Successfully added resource record set: {0xc0001a8c80 0xc0000105c0} commontests.go:70: Correctly replaced resource record {0xc0001a8c80 0xc0000105c0} -> {0xc0001a8d80 0xc000010610} commontests.go:142: Got 1 recordsets: [0xc00035b900] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00035bb20} route53_test.go:93: Got zone list: [0xc00035bb50] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000106d0} commontests.go:192: Successfully added resource record set: {0xc0001a9000 0xc0000106f0} commontests.go:142: Got 2 recordsets: [0xc00035bdc0 0xc00035bde0] commontests.go:142: Got 2 recordsets: [0xc00035be50 0xc00035be70] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036c010} route53_test.go:93: Got zone list: [0xc00036c040] route53_test.go:98: Got at least 1 zone in list:&{0xc000342480 0xc0000107f0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.038s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.046s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306920} clouddns_test.go:71: Got zone list: [0xc00051bb90] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc00051bb60} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306a20} clouddns_test.go:71: Got zone list: [0xc00051bc38] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc00051bc08} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306b20} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000322b80 0xc00051bce0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306be0} clouddns_test.go:71: Got zone list: [0xc00051bd88] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc00051bd58} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306d00} clouddns_test.go:71: Got zone list: [0xc00051bea8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc00051be78} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306e40} clouddns_test.go:71: Got zone list: [0xc000130018] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc00051bfc8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000306fa0} clouddns_test.go:71: Got zone list: [0xc0001302d0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc0001302a0} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000307100} clouddns_test.go:71: Got zone list: [0xc000130450] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc000130420} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000307240} clouddns_test.go:71: Got zone list: [0xc0001305a0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc000130570} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc0003073a0} clouddns_test.go:71: Got zone list: [0xc0001308b8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc000130888} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000307500} clouddns_test.go:71: Got zone list: [0xc000130f60] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc000130f18} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000307680} clouddns_test.go:71: Got zone list: [0xc0001315d8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc0001315a8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00055c690 0xc000307860} clouddns_test.go:71: Got zone list: [0xc000131bd8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000322a40 0xc000131b78} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.095s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.090s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.162s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0414 05:13:07.993376 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:07.993455 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:07.993500 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:07.993553 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:07.993594 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:07.993620 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:07.993649 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:07.993692 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:07.993714 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.004282 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0414 05:13:08.008419 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.008448 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.008471 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.008496 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.008516 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.008529 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.008544 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.008569 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.008583 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.009453 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I0414 05:13:08.012217 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.012242 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.012261 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.012281 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.012298 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.012311 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.012326 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.012340 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.012350 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.013119 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.014808 29719 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0414 05:13:08.014824 29719 assetstore.go:111] bin/containerd I0414 05:13:08.014831 29719 assetstore.go:111] bin/containerd-shim I0414 05:13:08.014837 29719 assetstore.go:111] bin/containerd-shim-runc-v1 I0414 05:13:08.014843 29719 assetstore.go:111] bin/containerd-shim-runc-v2 I0414 05:13:08.014848 29719 assetstore.go:111] bin/containerd-stress I0414 05:13:08.014854 29719 assetstore.go:111] bin/ctr I0414 05:13:08.014884 29719 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0414 05:13:08.014906 29719 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0414 05:13:08.017359 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.017387 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.017418 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.017441 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.017461 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.017474 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.017498 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.017528 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.017539 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.018328 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.019272 29719 containerd.go:57] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0414 05:13:08.020158 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.020188 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.020205 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.020223 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.020284 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.020297 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.020320 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.020332 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.020343 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.021123 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.022288 29719 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0414 05:13:08.023049 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.023073 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.023093 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.023116 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.023156 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.023172 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.023188 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.023205 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.023218 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.024052 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.025149 29719 assetstore.go:108] Matching assets for "^docker/": I0414 05:13:08.025163 29719 assetstore.go:111] docker/containerd I0414 05:13:08.025170 29719 assetstore.go:111] docker/containerd-shim I0414 05:13:08.025175 29719 assetstore.go:111] docker/ctr I0414 05:13:08.025180 29719 assetstore.go:111] docker/docker I0414 05:13:08.025185 29719 assetstore.go:111] docker/docker-init I0414 05:13:08.025191 29719 assetstore.go:111] docker/docker-proxy I0414 05:13:08.025196 29719 assetstore.go:111] docker/dockerd I0414 05:13:08.025201 29719 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.00s) === RUN TestDockerBuilder_LogFlags I0414 05:13:08.028078 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.028108 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.028129 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.028154 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.028196 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.028210 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.028225 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.028241 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.028254 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.030578 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.032187 29719 assetstore.go:108] Matching assets for "^docker/": I0414 05:13:08.032206 29719 assetstore.go:111] docker/containerd I0414 05:13:08.032215 29719 assetstore.go:111] docker/containerd-shim I0414 05:13:08.032222 29719 assetstore.go:111] docker/ctr I0414 05:13:08.032230 29719 assetstore.go:111] docker/docker I0414 05:13:08.032237 29719 assetstore.go:111] docker/docker-init I0414 05:13:08.032243 29719 assetstore.go:111] docker/docker-proxy I0414 05:13:08.032250 29719 assetstore.go:111] docker/dockerd I0414 05:13:08.032256 29719 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0414 05:13:08.035145 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.035178 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.035205 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.035235 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.035288 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.035305 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.035322 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.035343 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.035360 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.036309 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.037807 29719 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0414 05:13:08.038071 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.038096 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.038122 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.038160 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.038203 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.038222 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.038240 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.038257 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.038271 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.039186 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:08.040494 29719 assetstore.go:108] Matching assets for "^docker/": I0414 05:13:08.040512 29719 assetstore.go:111] docker/docker I0414 05:13:08.040520 29719 assetstore.go:111] docker/docker-containerd I0414 05:13:08.040526 29719 assetstore.go:111] docker/docker-containerd-ctr I0414 05:13:08.040533 29719 assetstore.go:111] docker/docker-containerd-shim I0414 05:13:08.040539 29719 assetstore.go:111] docker/docker-init I0414 05:13:08.040546 29719 assetstore.go:111] docker/docker-proxy I0414 05:13:08.040552 29719 assetstore.go:111] docker/docker-runc I0414 05:13:08.040558 29719 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0414 05:13:08.044582 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.044625 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.044652 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.044679 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.044702 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.044720 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.044738 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.044757 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.044772 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.046906 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0414 05:13:08.049224 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.049263 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.049288 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.049316 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.049339 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.049356 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.049372 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.049387 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.049401 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.051080 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0414 05:13:08.053350 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.053392 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.053415 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.053439 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.053461 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.053479 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.053494 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.053510 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.053525 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.056589 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0414 05:13:08.061674 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.061711 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.061731 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.061754 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.061773 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.061785 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.061801 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.061816 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.061829 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.063040 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0414 05:13:08.069480 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.069515 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.069533 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.069552 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.069569 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.069582 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.069595 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.069609 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.069620 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.070805 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0414 05:13:08.078448 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.078525 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.078554 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.078585 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.078611 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.078627 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.078645 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.078666 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.078681 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.080310 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0414 05:13:08.087636 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.087686 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.087717 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.087749 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.087774 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.087791 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.087810 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.087828 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.087844 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.089559 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0414 05:13:08.097436 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.097487 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:08.097515 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:08.097545 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:08.097571 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:08.097588 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:08.097605 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:08.097626 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:08.097643 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:08.099327 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.36s) === RUN TestKubeAPIServerBuilderARM64 I0414 05:13:09.461728 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.461790 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.461817 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.461847 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.461873 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.461890 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.461907 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.461927 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.461944 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.463533 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.07s) === RUN TestKubeControllerManagerBuilder I0414 05:13:09.533857 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.533910 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.533939 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.533986 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.534014 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.534030 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.534047 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.534066 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.534082 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.535729 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0414 05:13:09.541791 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.541833 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.541857 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.541883 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.541906 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.541922 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.541939 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.541956 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.541971 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.543464 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0414 05:13:09.614229 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.614310 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.614366 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.614420 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.614467 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.614503 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.614536 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.614571 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.614599 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.617066 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0414 05:13:09.688194 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.688264 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.688308 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.688356 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.688393 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.688420 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.688448 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.688478 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.688501 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.691219 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0414 05:13:09.697562 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.697635 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.697682 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.697733 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.697776 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.697807 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.697840 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.697873 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.697900 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.700869 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0414 05:13:09.763932 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.764008 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.764045 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.764077 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.764106 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.764123 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.764140 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.764162 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.764178 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.765883 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.06s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0414 05:13:09.828085 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.828137 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.828164 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.828196 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.828222 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.828239 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.828255 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.828275 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.828291 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.831645 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0414 05:13:09.835982 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.836027 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.836054 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.836082 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.836105 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.836121 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.836139 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.836158 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.836171 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.837747 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.06s) === RUN TestKubeSchedulerBuilderARM64 I0414 05:13:09.899945 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.900017 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.900063 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.900108 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.900145 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.900173 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.900200 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.900230 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.900253 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.902099 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.06s) === RUN TestKubectlBuilder I0414 05:13:09.963295 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.963348 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.963375 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.963407 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.963433 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.963448 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.963465 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.963484 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.963500 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.966815 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.968602 29719 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0414 05:13:09.970215 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.970261 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.970302 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.970344 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.970375 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.970403 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.970434 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.970463 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.970488 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.971721 29719 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.00s) === RUN Test_RunKubeletBuilderWarmPool I0414 05:13:09.975089 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.975133 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.975170 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.975214 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.975245 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.975276 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.975307 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.975336 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.975363 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.976532 29719 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0414 05:13:09.980270 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.980318 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.980356 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.980402 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.980433 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.980462 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.980492 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.980522 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.980546 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.982064 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.983565 29719 assetstore.go:108] Matching assets for "protokube$": I0414 05:13:09.983586 29719 assetstore.go:111] /opt/kops/bin/protokube I0414 05:13:09.983601 29719 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0414 05:13:09.983635 29719 assetstore.go:108] Matching assets for "channels$": I0414 05:13:09.983651 29719 assetstore.go:111] /opt/kops/bin/channels I0414 05:13:09.983663 29719 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I0414 05:13:09.985067 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.985110 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.985148 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.985196 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.985228 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.985258 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.985290 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.985317 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.985342 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.986808 29719 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0414 05:13:09.989013 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.989059 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.989096 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.989138 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.989170 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.989201 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.989232 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.989260 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.989284 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.990786 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.992318 29719 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0414 05:13:09.992798 29719 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.992842 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.992883 29719 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.992926 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.992957 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.992987 29719 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.993018 29719 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.993047 29719 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.993071 29719 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.994524 29719 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.996000 29719 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.061s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.091s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.063s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.065s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.041s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0414 05:12:52.042154 25484 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0414 05:12:52.042242 25484 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.030s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0414 05:13:00.975119 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975239 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975285 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975317 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975346 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0414 05:13:00.975530 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975570 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975598 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975626 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975659 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975686 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975719 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975762 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975795 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975834 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975864 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975894 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975921 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975958 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.975983 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.976034 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.976064 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.976089 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0414 05:13:00.976116 28088 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0414 05:13:00.976233 28088 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0414 05:13:00.976285 28088 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvdia --- PASS: Test_Validate_Nvdia (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.187s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0414 05:12:53.327204 25862 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0414 05:12:53.327254 25862 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0414 05:12:53.327264 25862 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0414 05:12:53.327273 25862 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.078s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.025s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.01s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.091s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.066s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.03s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.203s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.081s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.036s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.017s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.03s) PASS ok k8s.io/kops/pkg/edit 0.143s === RUN TestFlagToFalse I0414 05:12:51.914935 25426 featureflag_test.go:33] Created flag Unittest1 I0414 05:12:51.915008 25426 featureflag.go:162] FeatureFlag "UnitTest1"=false I0414 05:12:51.915023 25426 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0414 05:12:51.915108 25426 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.024s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.113s === RUN TestRenderInstanceGroupZones W0414 05:12:54.858442 26306 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.077s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0414 05:13:21.452358 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.452418 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.453584 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.453603 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.458818 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.458837 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.458847 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.458855 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.460138 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.460656 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.460675 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.460687 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.461864 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.461891 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.467117 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.467141 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:21.467606 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:21.467623 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.467635 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.468791 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.468817 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.473999 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.474035 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.474052 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.474064 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.475048 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.475538 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.475566 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.475581 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.476765 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.476793 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.482061 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.482093 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.482730 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.482758 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.482775 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.483975 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.484021 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.489157 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.489202 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.489889 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.489923 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.489940 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.491119 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.491155 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.496442 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.496473 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.496504 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.496517 29992 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0414 05:13:21.497767 29992 instancegroups.go:398] Draining the node: "node-2a.local". I0414 05:13:21.498788 29992 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0414 05:13:21.498819 29992 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0414 05:13:21.498838 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.500034 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.500104 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.505412 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.505456 29992 instancegroups.go:398] Draining the node: "node-2b.local". I0414 05:13:21.506335 29992 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0414 05:13:21.506382 29992 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0414 05:13:21.506402 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.507596 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.507632 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.512932 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.512978 29992 instancegroups.go:398] Draining the node: "node-2c.local". I0414 05:13:21.513924 29992 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0414 05:13:21.513976 29992 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0414 05:13:21.514020 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.515223 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.515259 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.520520 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.520581 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0414 05:13:21.521231 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.521257 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.522428 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.522479 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.522516 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.522528 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.522556 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.523728 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.523772 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.523790 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.523812 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.525000 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.525036 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.525083 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.525097 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.525120 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.526310 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.526361 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.526376 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.526401 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.527587 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.527638 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.527652 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.527677 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.528875 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.528899 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.528945 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.528963 29992 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0414 05:13:21.528987 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.530177 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.530228 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.530243 29992 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0414 05:13:21.530268 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.531437 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0414 05:13:21.531484 29992 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0414 05:13:21.531498 29992 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0414 05:13:21.531520 29992 instancegroups.go:435] waiting for 1ms after terminating instance W0414 05:13:21.532696 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0414 05:13:21.532724 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0414 05:13:21.533083 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.533111 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.534286 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.534322 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.534358 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.534407 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.534428 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.534440 29992 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.534451 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.535203 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.536030 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.536060 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.536080 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.537250 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.537284 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.537319 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.537373 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:21.538204 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:21.538241 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.538263 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.539428 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.539459 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.539476 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.539493 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.539507 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.539517 29992 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.539528 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.540483 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.541296 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.541326 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.541344 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.542528 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.542562 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.542583 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.542617 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.543362 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.543389 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.543409 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.544566 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.544596 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.544612 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.544644 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.545379 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.545417 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.545436 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.546607 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.546652 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.546670 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.546681 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.546695 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.546706 29992 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.546717 29992 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0414 05:13:21.547593 29992 instancegroups.go:398] Draining the node: "node-2a.local". I0414 05:13:21.548259 29992 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0414 05:13:21.548283 29992 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0414 05:13:21.548299 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.549457 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.549485 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.549523 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.549554 29992 instancegroups.go:398] Draining the node: "node-2b.local". I0414 05:13:21.552319 29992 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0414 05:13:21.552360 29992 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0414 05:13:21.552385 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.553509 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.553562 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.553597 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.553653 29992 instancegroups.go:398] Draining the node: "node-2c.local". I0414 05:13:21.554656 29992 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0414 05:13:21.554697 29992 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0414 05:13:21.554732 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.555928 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.555990 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0414 05:13:21.556018 29992 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0414 05:13:21.556038 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0414 05:13:21.556314 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0414 05:13:21.556592 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.556624 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.557761 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.557801 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.563078 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.563140 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.563164 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.563177 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.563944 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.564780 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.564814 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.564838 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.566005 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.566044 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.571319 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.571368 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:21.572233 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:21.572267 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.572295 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.573453 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.573505 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.578795 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.578834 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.578857 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.578873 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.579950 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.580821 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.580859 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.580882 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.582083 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.582118 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.587356 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.587408 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.588345 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.588380 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.588404 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.589568 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.589609 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.594916 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.594966 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.595925 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.595959 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.595983 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.597173 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.597207 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.602452 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.602487 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.602506 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.602520 29992 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0414 05:13:21.603831 29992 instancegroups.go:398] Draining the node: "node-2a.local". I0414 05:13:21.604924 29992 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0414 05:13:21.604964 29992 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0414 05:13:21.604989 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.606161 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.606208 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.611462 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.611516 29992 instancegroups.go:398] Draining the node: "node-2b.local". I0414 05:13:21.612457 29992 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0414 05:13:21.612491 29992 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0414 05:13:21.612514 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.613674 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.613719 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.619024 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.619080 29992 instancegroups.go:398] Draining the node: "node-2c.local". I0414 05:13:21.620078 29992 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0414 05:13:21.620115 29992 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0414 05:13:21.620138 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.621302 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.621348 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.626605 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.626638 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0414 05:13:21.626802 29992 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0414 05:13:21.627419 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.627451 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.628648 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.628688 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.628708 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0414 05:13:21.629096 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.629127 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.630320 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.630355 29992 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0414 05:13:21.630378 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0414 05:13:21.630664 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.630696 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.630711 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0414 05:13:21.630897 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.630924 29992 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0414 05:13:21.630939 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0414 05:13:21.631213 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.631243 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.632433 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.632474 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.637732 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.637799 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.637823 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.637836 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.638713 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.639498 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.639532 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.639555 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.640707 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.640755 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.640800 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0414 05:13:21.641084 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.641112 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.642314 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.642354 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.647599 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.647702 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.647739 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.647760 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.648897 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.649773 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.649809 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.649833 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.650971 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.651013 29992 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0414 05:13:21.651034 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0414 05:13:21.651331 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.651359 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.651372 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.652412 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.653177 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.653210 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.653234 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.654357 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.654385 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.659619 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.659669 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.660564 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.660601 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.660646 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.661852 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.661888 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.667134 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.667184 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.668031 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.668065 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.668088 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.669282 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.669317 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.674567 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.674600 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0414 05:13:21.674920 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.674950 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.674967 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0414 05:13:21.675275 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.675307 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.675323 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0414 05:13:21.675588 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.675617 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.675633 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0414 05:13:21.675818 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.675847 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.675860 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.676988 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.677777 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.677813 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.677835 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.679021 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.679067 29992 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0414 05:13:21.679088 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0414 05:13:21.679297 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.679329 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.679342 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.680422 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.681241 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.681275 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.681298 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.682497 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.682539 29992 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0414 05:13:21.682562 29992 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0414 05:13:21.682818 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.682848 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.684028 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.684075 29992 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0414 05:13:21.685451 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.690697 29992 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0414 05:13:21.691872 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.697126 29992 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0414 05:13:21.698298 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.703546 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.703615 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.703641 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.703654 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.704416 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.705209 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.705244 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.705267 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.706463 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.706509 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.711773 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.711853 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:21.713053 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:21.713104 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.713139 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.714326 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.714396 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.719721 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.719761 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.719788 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.719803 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.720809 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.721561 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.721596 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.721619 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.722811 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.722862 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.728151 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.728209 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.729022 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.729060 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.729083 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.730282 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.730329 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.735553 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.735611 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.736470 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.736504 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.736527 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.737715 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.737764 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.743048 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.743081 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.743108 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.743147 29992 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0414 05:13:21.744120 29992 instancegroups.go:398] Draining the node: "node-2a.local". I0414 05:13:21.744873 29992 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0414 05:13:21.744906 29992 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0414 05:13:21.744929 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.746119 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.746168 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.751408 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.751468 29992 instancegroups.go:398] Draining the node: "node-2b.local". I0414 05:13:21.752289 29992 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0414 05:13:21.752324 29992 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0414 05:13:21.752347 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.753548 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.753598 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.758891 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.758950 29992 instancegroups.go:398] Draining the node: "node-2c.local". I0414 05:13:21.759758 29992 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0414 05:13:21.759793 29992 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0414 05:13:21.759816 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.760995 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.761048 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.766308 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.766343 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0414 05:13:21.766649 29992 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0414 05:13:21.766678 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.767862 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.767905 29992 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0414 05:13:21.769030 29992 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0414 05:13:21.770182 29992 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0414 05:13:21.771327 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.776518 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.776631 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.776669 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.776686 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.777614 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.778588 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.778622 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.778659 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.779861 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.779905 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.785113 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.785164 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:21.786127 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:21.786162 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.786184 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.787360 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.787396 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.792622 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.792663 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.792684 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.792699 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.793902 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.794776 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.794813 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.794837 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.796016 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.796052 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.801301 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.801354 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.802227 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.802262 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.802285 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.803481 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.803517 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.808794 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.808872 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.809756 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.809790 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.809813 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.810996 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.811032 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.816270 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.816302 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.816323 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.816338 29992 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0414 05:13:21.817394 29992 instancegroups.go:398] Draining the node: "node-2a.local". I0414 05:13:21.818182 29992 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0414 05:13:21.818219 29992 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0414 05:13:21.818243 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.819438 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.819473 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.824717 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.824785 29992 instancegroups.go:398] Draining the node: "node-2b.local". I0414 05:13:21.825708 29992 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0414 05:13:21.825744 29992 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0414 05:13:21.825767 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.826930 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.826983 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.832213 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.832279 29992 instancegroups.go:398] Draining the node: "node-2c.local". I0414 05:13:21.833145 29992 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0414 05:13:21.833180 29992 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0414 05:13:21.833203 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.834355 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.834410 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.839667 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.839708 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0414 05:13:21.840393 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.840422 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.840434 29992 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0414 05:13:21.841226 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.842034 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.842067 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.842090 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.843252 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.843292 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.848548 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.848611 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.849447 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.849483 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.849505 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.850684 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.850737 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.855985 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.856023 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0414 05:13:21.856290 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.856318 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.856332 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.857057 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:21.857875 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:21.857913 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:21.857936 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.859108 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.859146 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.864393 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.864456 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:21.865272 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:21.865305 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:21.865348 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.866515 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.866553 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.871773 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.871810 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0414 05:13:21.872118 29992 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled W0414 05:13:21.872157 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0414 05:13:21.872177 29992 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled W0414 05:13:21.872193 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0414 05:13:21.872212 29992 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled W0414 05:13:21.872222 29992 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0414 05:13:21.872239 29992 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0414 05:13:21.872253 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0414 05:13:21.872470 29992 instancegroups.go:568] Detaching instance "bastion-1a", in group "bastion-1". I0414 05:13:21.872517 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:21.873677 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.873721 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.878961 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.879003 29992 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled I0414 05:13:21.879055 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.879078 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.879091 29992 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0414 05:13:21.879788 29992 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled I0414 05:13:21.879816 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.879835 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.879848 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:21.880742 29992 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0414 05:13:21.880795 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:21.881954 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.881996 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.887510 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.887556 29992 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled I0414 05:13:21.887585 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.887608 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.887622 29992 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0414 05:13:21.888620 29992 instancegroups.go:566] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0414 05:13:21.888674 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:21.889875 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.889909 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:21.895112 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.895146 29992 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0414 05:13:21.895163 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0414 05:13:21.895388 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.895420 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.895434 29992 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0414 05:13:21.897591 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:21.898336 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:21.898367 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:21.898400 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.899607 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.899653 29992 instancegroups.go:501] Cluster validated. I0414 05:13:21.899698 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:21.899771 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:21.901067 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:21.901102 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:21.901438 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:21.901474 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:21.901505 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:21.902701 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:21.902768 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.003376 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.003526 29992 instancegroups.go:398] Draining the node: "node-1e.local". I0414 05:13:22.003539 29992 instancegroups.go:398] Draining the node: "node-1d.local". I0414 05:13:22.004758 29992 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0414 05:13:22.004785 29992 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0414 05:13:22.005120 29992 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0414 05:13:22.005156 29992 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0414 05:13:22.005192 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.006392 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.006438 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.006478 29992 instancegroups.go:398] Draining the node: "node-1f.local". I0414 05:13:22.007444 29992 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0414 05:13:22.007475 29992 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0414 05:13:22.007504 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.008724 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.008795 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.109273 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.109337 29992 instancegroups.go:398] Draining the node: "node-1g.local". I0414 05:13:22.110203 29992 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0414 05:13:22.110237 29992 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0414 05:13:22.110270 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.111483 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.111528 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.111547 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0414 05:13:22.113568 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.113643 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.113658 29992 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0414 05:13:22.116233 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:22.116273 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:22.117205 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:22.117229 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:22.117552 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:22.117591 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:22.117625 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.118839 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.118907 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.219334 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.219430 29992 instancegroups.go:398] Draining the node: "node-1d.local". I0414 05:13:22.219546 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:22.220991 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:22.221032 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:22.221358 29992 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0414 05:13:22.221399 29992 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0414 05:13:22.221438 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.222647 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.222693 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.222733 29992 instancegroups.go:398] Draining the node: "node-1e.local". I0414 05:13:22.223587 29992 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0414 05:13:22.223621 29992 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0414 05:13:22.223651 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.224847 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.224917 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.325464 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.325539 29992 instancegroups.go:398] Draining the node: "node-1f.local". I0414 05:13:22.326683 29992 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0414 05:13:22.326720 29992 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0414 05:13:22.326759 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.327958 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.328006 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.328025 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0414 05:13:22.328311 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.328348 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.328363 29992 instancegroups.go:309] Tainting 7 nodes in "master-1" instancegroup. I0414 05:13:22.330655 29992 instancegroups.go:398] Draining the node: "master-1a.local". I0414 05:13:22.331443 29992 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0414 05:13:22.331478 29992 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0414 05:13:22.331511 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.332721 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.332766 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.332809 29992 instancegroups.go:398] Draining the node: "master-1c.local". I0414 05:13:22.332878 29992 instancegroups.go:398] Draining the node: "master-1b.local". I0414 05:13:22.334174 29992 instancegroups.go:417] deleting node "master-1c.local" from kubernetes I0414 05:13:22.334211 29992 instancegroups.go:589] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0414 05:13:22.334550 29992 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0414 05:13:22.334588 29992 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0414 05:13:22.334624 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.335829 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.335897 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.436563 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.436628 29992 instancegroups.go:398] Draining the node: "master-1e.local". I0414 05:13:22.436701 29992 instancegroups.go:398] Draining the node: "master-1d.local". I0414 05:13:22.437989 29992 instancegroups.go:417] deleting node "master-1e.local" from kubernetes I0414 05:13:22.438071 29992 instancegroups.go:589] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0414 05:13:22.438390 29992 instancegroups.go:417] deleting node "master-1d.local" from kubernetes I0414 05:13:22.438433 29992 instancegroups.go:589] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0414 05:13:22.438465 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.439676 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.439721 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.439762 29992 instancegroups.go:398] Draining the node: "master-1f.local". I0414 05:13:22.440585 29992 instancegroups.go:417] deleting node "master-1f.local" from kubernetes I0414 05:13:22.440644 29992 instancegroups.go:589] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0414 05:13:22.440677 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.441881 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.441949 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.542526 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.542582 29992 instancegroups.go:398] Draining the node: "master-1g.local". I0414 05:13:22.543510 29992 instancegroups.go:417] deleting node "master-1g.local" from kubernetes I0414 05:13:22.543547 29992 instancegroups.go:589] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0414 05:13:22.543580 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.544789 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.544836 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.544854 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0414 05:13:22.545115 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.545150 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.545165 29992 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0414 05:13:22.546921 29992 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0414 05:13:22.546987 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:22.548186 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.548232 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.548246 29992 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0414 05:13:22.548305 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:22.549505 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.549553 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.549597 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:22.549675 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:22.550920 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:22.550955 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:22.551291 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:22.551331 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:22.551365 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.552571 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.552638 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.653085 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.653145 29992 instancegroups.go:398] Draining the node: "node-1d.local". I0414 05:13:22.653204 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:22.654476 29992 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0414 05:13:22.654516 29992 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0414 05:13:22.654792 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:22.654825 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:22.654853 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.656008 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.656051 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.656090 29992 instancegroups.go:398] Draining the node: "node-1e.local". I0414 05:13:22.656988 29992 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0414 05:13:22.657022 29992 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0414 05:13:22.657057 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.658269 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.658334 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.758821 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.758884 29992 instancegroups.go:398] Draining the node: "node-1f.local". I0414 05:13:22.759713 29992 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0414 05:13:22.759748 29992 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0414 05:13:22.759783 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.760936 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.760968 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.760979 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0414 05:13:22.761166 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.761186 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.761194 29992 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0414 05:13:22.762224 29992 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0414 05:13:22.762251 29992 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0414 05:13:22.762274 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:22.763471 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.763517 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.763563 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:22.763634 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:22.764901 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:22.764962 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:22.765279 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:22.765317 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:22.765349 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.766501 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.766540 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.866911 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.866973 29992 instancegroups.go:398] Draining the node: "node-1d.local". I0414 05:13:22.867049 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:22.868314 29992 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0414 05:13:22.868353 29992 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0414 05:13:22.868629 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:22.868661 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:22.868690 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.869842 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.869866 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.869888 29992 instancegroups.go:398] Draining the node: "node-1e.local". I0414 05:13:22.870378 29992 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0414 05:13:22.870395 29992 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0414 05:13:22.870414 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.871606 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.871676 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.972341 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.972368 29992 instancegroups.go:398] Draining the node: "node-1f.local". I0414 05:13:22.972850 29992 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0414 05:13:22.972868 29992 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0414 05:13:22.972887 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.974075 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.974121 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.974138 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0414 05:13:22.974362 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.974390 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.974403 29992 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0414 05:13:22.975088 29992 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0414 05:13:22.975121 29992 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0414 05:13:22.975155 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:22.976303 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.976323 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:22.981490 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.981521 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:22.981586 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:22.982346 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:22.982371 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:22.982386 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.982546 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:22.982564 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:22.982576 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.983586 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.983622 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:22.988803 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.988821 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0414 05:13:22.988946 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.988961 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.988968 29992 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0414 05:13:22.989368 29992 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0414 05:13:22.989392 29992 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0414 05:13:22.989414 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:22.990605 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.990659 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:22.995884 29992 instancegroups.go:501] Cluster validated. I0414 05:13:22.995918 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:22.995970 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:22.996740 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:22.996760 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:22.996774 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.996931 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:22.996947 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:22.996959 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:22.997970 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:22.998006 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:23.003208 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.003227 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0414 05:13:23.003357 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.003376 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.003384 29992 instancegroups.go:309] Tainting 4 nodes in "node-1" instancegroup. I0414 05:13:23.004101 29992 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0414 05:13:23.004134 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:23.005464 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.005510 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.005524 29992 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0414 05:13:23.005573 29992 instancegroups.go:170] waiting for 1ms after detaching instance I0414 05:13:23.006722 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.006747 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.006769 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:23.006808 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:23.006865 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:23.007860 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:23.007880 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:23.007899 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.008070 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:23.008088 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:23.008104 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.008241 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:23.008257 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:23.008272 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.009100 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.109551 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.109579 29992 instancegroups.go:398] Draining the node: "node-1d.local". I0414 05:13:23.110059 29992 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0414 05:13:23.110077 29992 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0414 05:13:23.110095 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.111294 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.111339 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.111357 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0414 05:13:23.111586 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.111618 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.111632 29992 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0414 05:13:23.113859 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:23.114654 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:23.114689 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:23.114719 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.115869 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.115894 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.115921 29992 instancegroups.go:398] Draining the node: "node-1e.local". I0414 05:13:23.115960 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:23.116716 29992 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0414 05:13:23.116736 29992 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0414 05:13:23.116895 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:23.116911 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:23.116927 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.118200 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.118272 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.218634 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.218664 29992 instancegroups.go:398] Draining the node: "node-1g.local". I0414 05:13:23.218721 29992 instancegroups.go:398] Draining the node: "node-1f.local". I0414 05:13:23.219481 29992 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0414 05:13:23.219502 29992 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0414 05:13:23.219659 29992 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0414 05:13:23.219676 29992 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0414 05:13:23.219692 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.220880 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.220927 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.220969 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:23.221797 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:23.221831 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:23.221861 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.223058 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.223093 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.323513 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.323580 29992 instancegroups.go:398] Draining the node: "node-1d.local". I0414 05:13:23.324295 29992 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0414 05:13:23.324319 29992 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0414 05:13:23.324347 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.325539 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.325587 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.325602 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0414 05:13:23.325821 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.325844 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.325860 29992 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0414 05:13:23.325878 29992 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0414 05:13:23.325892 29992 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0414 05:13:23.325906 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0414 05:13:23.326027 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.326048 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.326059 29992 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0414 05:13:23.326879 29992 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0414 05:13:23.326909 29992 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0414 05:13:23.326924 29992 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0414 05:13:23.326957 29992 instancegroups.go:398] Draining the node: "node-1a.local". I0414 05:13:23.327596 29992 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0414 05:13:23.327625 29992 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0414 05:13:23.327646 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.328794 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.328817 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:23.333998 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.334055 29992 instancegroups.go:398] Draining the node: "node-1b.local". I0414 05:13:23.334759 29992 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0414 05:13:23.334779 29992 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0414 05:13:23.334793 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.336172 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.336198 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:23.341374 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.341402 29992 instancegroups.go:398] Draining the node: "node-1c.local". I0414 05:13:23.341876 29992 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0414 05:13:23.341894 29992 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0414 05:13:23.341907 29992 instancegroups.go:435] waiting for 1ms after terminating instance I0414 05:13:23.343072 29992 instancegroups.go:468] Validating the cluster. I0414 05:13:23.343098 29992 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0414 05:13:23.348293 29992 instancegroups.go:501] Cluster validated. I0414 05:13:23.348313 29992 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.964s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.028s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.059s === RUN TestParse E0414 05:12:53.462007 25959 versions.go:96] unable to parse Kubernetes version "" E0414 05:12:53.462072 25959 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.048s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.061s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.19s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.01s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) PASS ok k8s.io/kops/pkg/kubeconfig 0.311s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.074s ? k8s.io/kops/pkg/model/alimodel [no test files] === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.048s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.056s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.116s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/pollinterval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0414 05:13:05.207978 29324 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0414 05:13:05.208084 29324 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0414 05:13:05.209016 29324 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0414 05:13:05.209128 29324 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.06s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/pollinterval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.00s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.157s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.067s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.091s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.056s === RUN TestServerGroupBuilder --- PASS: TestServerGroupBuilder (0.22s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.272s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.065s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.039s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.041s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.51s) --- PASS: TestIssueCert/ca (0.44s) --- PASS: TestIssueCert/client (0.20s) --- PASS: TestIssueCert/clientOneYear (0.87s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTrip --- PASS: TestPrivateKeyRoundTrip (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.538s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.030s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/ali [no test files] === RUN TestAddUntaggedRouteTables I0414 05:13:05.841707 29534 routetable.go:53] DescribeRouteTables: { } I0414 05:13:05.841787 29534 aws.go:224] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W0414 05:13:05.841808 29534 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0414 05:13:05.841879 29534 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0414 05:13:05.841951 29534 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0414 05:13:05.842016 29534 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0414 05:13:05.842083 29534 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0414 05:13:05.842107 29534 volumes.go:128] DescribeVolumes: { } W0414 05:13:05.842132 29534 aws.go:2236] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0414 05:13:05.842356 29534 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:05.842387 29534 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:05.842435 29534 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:05.842489 29534 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:05.842518 29534 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:05.842533 29534 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:05.842558 29534 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:05.842585 29534 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:05.842605 29534 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:05.842629 29534 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0414 05:13:05.842651 29534 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0414 05:13:05.842682 29534 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0414 05:13:05.842707 29534 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0414 05:13:05.842723 29534 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0414 05:13:05.842753 29534 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0414 05:13:05.842789 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0414 05:13:05.842817 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0414 05:13:05.842836 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0414 05:13:05.842853 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0414 05:13:05.842872 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0414 05:13:05.842903 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0414 05:13:05.842931 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0414 05:13:05.842949 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0414 05:13:05.842967 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0414 05:13:05.842983 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0414 05:13:05.843001 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0414 05:13:05.843016 29534 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.050s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.079s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.082s ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.030s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.031s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.033s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.016s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.01s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.105s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.067s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/ali [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.031s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0414 05:12:57.501404 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501468 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501479 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501488 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501497 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501505 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501513 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501522 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501531 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501540 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501549 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501568 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501579 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501588 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.501596 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.501605 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.504749 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.504798 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.504868 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.504880 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.504934 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.504946 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.504999 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505011 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505065 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505076 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505133 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505145 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505200 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505212 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505282 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505295 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505350 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505362 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505415 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505428 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505481 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505493 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505555 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505568 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505621 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505633 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505688 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505700 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505758 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505771 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505830 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505843 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505902 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505914 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.505966 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.505979 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506034 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506060 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506117 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506130 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506187 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506199 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506257 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506270 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506324 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506337 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506394 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506405 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506459 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506472 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506529 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506542 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506612 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506626 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506681 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506693 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506747 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506759 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506814 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506826 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506880 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506894 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.506949 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.506964 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507035 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507049 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507104 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507117 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507169 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507180 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507236 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507248 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507324 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507338 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507395 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507407 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507463 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507475 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507530 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507542 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507597 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507609 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507663 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507675 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507747 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507760 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.507816 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.507828 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508068 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.508118 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508213 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.508256 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508356 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.508400 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508499 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.508542 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508682 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.508727 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508818 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.508883 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.508975 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509019 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509125 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509188 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509282 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509324 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509421 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509466 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509557 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509599 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509696 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509740 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509832 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.509873 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.509965 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510007 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510099 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510143 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510232 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510274 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510363 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510404 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510494 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510535 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510644 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510689 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510783 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510825 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.510922 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.510963 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511052 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511093 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511180 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511220 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511310 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511354 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511446 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511489 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511602 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511647 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511738 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511780 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.511869 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.511912 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512015 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512059 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512149 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512204 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512299 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512340 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512436 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512479 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512568 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512609 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512702 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512742 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512834 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.512876 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.512967 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513008 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513127 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513169 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513260 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513301 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513390 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513432 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513520 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513563 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513661 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513703 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513795 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513838 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.513928 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.513982 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514085 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514129 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514226 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514267 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514358 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514402 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514493 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514536 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514625 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514665 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514760 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514805 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.514898 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.514939 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.515028 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.515068 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.515161 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.515203 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.515297 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.515340 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.515446 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.515488 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.515578 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.515618 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0414 05:12:57.516359 27477 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0414 05:12:57.516402 27477 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0414 05:12:57.516430 27477 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0414 05:12:57.516455 27477 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0414 05:12:57.516489 27477 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0414 05:12:57.516522 27477 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0414 05:12:57.516554 27477 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0414 05:12:57.516613 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516653 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516681 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.516708 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516742 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.516778 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516805 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.516832 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516866 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.516900 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516927 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516952 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.516987 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.517025 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.517055 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.517083 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.525664 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.525701 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.525771 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.525784 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.525846 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.525858 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.525924 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.525937 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.525997 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526010 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526064 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526076 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526130 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526142 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526198 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526211 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526265 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526278 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526332 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526344 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526399 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526412 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526468 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526481 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526561 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526574 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526629 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526643 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526702 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526714 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526768 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526781 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526835 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526848 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526901 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526913 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.526975 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.526988 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527043 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527062 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527117 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527129 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527181 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527193 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527247 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527260 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527338 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527353 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527409 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527421 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527477 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527489 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527542 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527554 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527610 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527622 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527709 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527724 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527780 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527793 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527846 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527857 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527911 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527925 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.527978 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.527990 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528058 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528071 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528127 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528139 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528195 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528208 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528263 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528276 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528330 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528342 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528398 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528410 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528464 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528477 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528530 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528542 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528594 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528620 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528688 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528701 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528764 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528777 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528830 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528842 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528905 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528917 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.528970 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.528983 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529049 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529062 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529157 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529170 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529222 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529234 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529296 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529308 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529365 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529377 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529430 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529442 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529495 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529507 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529562 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529575 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529628 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529640 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529691 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529704 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529757 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529768 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529821 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529835 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529889 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529902 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.529956 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.529970 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530025 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530037 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530088 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530102 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530158 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530170 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530222 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530233 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530294 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530307 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530365 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530378 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530433 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530446 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530497 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530510 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530563 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530575 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530854 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.530892 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.530986 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531031 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531120 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531163 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531256 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531299 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531391 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531433 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531527 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531570 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531664 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531707 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531797 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531839 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.531928 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.531969 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532058 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532104 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532194 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532237 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532327 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532369 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532469 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532513 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532635 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532681 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532776 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532819 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.532911 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.532955 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533047 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533090 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533184 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533226 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533366 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533411 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533510 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533552 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533662 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533707 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533800 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533843 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.533936 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.533990 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.534083 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.534124 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.534212 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.534252 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.534342 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.534384 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.534472 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.534512 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.534603 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.534647 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0414 05:12:57.534741 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0414 05:12:57.534783 27477 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.042s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.042s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.04s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.127s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.101s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0414 05:13:02.247576 28398 files.go:57] Writing file "/tmp/fitest2980889926/SubDir/test1.tmp" I0414 05:13:02.247768 28398 files.go:113] Changing file mode for "/tmp/fitest2980889926/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.099s === RUN TestBootstrapChannelBuilder_BuildTasks I0414 05:13:09.553981 29787 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:09.554106 29787 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:09.554155 29787 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:09.554206 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:09.554240 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:09.554257 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:09.554286 29787 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:09.554314 29787 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:09.554329 29787 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:09.562425 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.627312 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.721724 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.806083 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.888786 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:09.956161 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:10.049375 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:10.132981 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:10.222297 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:10.308225 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:11.282063 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:12.479224 29787 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (2.99s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0414 05:13:12.539513 29787 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:12.539567 29787 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:12.539600 29787 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:12.539630 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:12.539655 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:12.539672 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:12.539691 29787 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:12.539711 29787 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:12.539726 29787 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:12.539785 29787 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0414 05:13:12.541580 29787 zones.go:93] ListHostedZonesPages { } I0414 05:13:12.603807 29787 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.06s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0414 05:13:12.604152 29787 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:12.604220 29787 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0414 05:13:12.604251 29787 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0414 05:13:12.604283 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0414 05:13:12.604308 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0414 05:13:12.604327 29787 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0414 05:13:12.604346 29787 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0414 05:13:12.604368 29787 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0414 05:13:12.604385 29787 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0414 05:13:12.605471 29787 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.09s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.17s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.12s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0414 05:13:13.003578 29787 new_cluster.go:935] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Kubenet --- PASS: TestPopulateCluster_Kubenet (0.00s) === RUN TestPopulateCluster_CNI --- PASS: TestPopulateCluster_CNI (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0414 05:13:13.035472 29787 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0414 05:13:13.035498 29787 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0414 05:13:13.035509 29787 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0414 05:13:13.035528 29787 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0414 05:13:13.035539 29787 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet a I0414 05:13:13.035558 29787 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0414 05:13:13.035577 29787 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet b I0414 05:13:13.035596 29787 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0414 05:13:13.035608 29787 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0414 05:13:13.035627 29787 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0414 05:13:13.035638 29787 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0414 05:13:13.035648 29787 subnets.go:182] Assigned CIDR 10.96.0.0/11 to subnet b I0414 05:13:13.035658 29787 subnets.go:182] Assigned CIDR 10.128.0.0/11 to subnet c I0414 05:13:13.035668 29787 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet c I0414 05:13:13.035679 29787 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0414 05:13:13.035690 29787 subnets.go:196] Assigned CIDR 10.4.0.0/14 to subnet b I0414 05:13:13.035700 29787 subnets.go:196] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 3.557s ? k8s.io/kops/upup/pkg/fi/cloudup/alitasks [no test files] === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/aliup 0.081s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0414 05:13:08.150547 29748 target.go:112] Terraform output is in /tmp/kops-render-3257140199 I0414 05:13:08.151763 29748 target.go:112] Terraform output is in /tmp/kops-render-3257140199 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I0414 05:13:08.152485 29748 target.go:168] Cloudformation output is in /tmp/kops-render-3076426566 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0414 05:13:08.152925 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0414 05:13:08.152996 29748 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0414 05:13:08.153039 29748 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0414 05:13:08.153228 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.153292 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.153340 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.153354 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.153467 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.153499 29748 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0414 05:13:08.153533 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.153572 29748 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0414 05:13:08.153633 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0414 05:13:08.153737 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.153768 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.153798 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.153810 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.153833 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.153872 29748 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0414 05:13:08.153920 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0414 05:13:08.154160 29748 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0414 05:13:08.154199 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.154265 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0414 05:13:08.154293 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.154319 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.154342 29748 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0414 05:13:08.154443 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.154572 29748 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0414 05:13:08.154609 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.154627 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.154647 29748 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0414 05:13:08.154682 29748 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.154786 29748 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0414 05:13:08.154825 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.154842 29748 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0414 05:13:08.154869 29748 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0414 05:13:08.154989 29748 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0414 05:13:08.155026 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.155058 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.155069 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.155086 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.155108 29748 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0414 05:13:08.155138 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.155177 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.155196 29748 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0414 05:13:08.155219 29748 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.155252 29748 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0414 05:13:08.155270 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0414 05:13:08.155296 29748 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0414 05:13:08.155400 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0414 05:13:08.155425 29748 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0414 05:13:08.155441 29748 internetgateways.go:66] CreateInternetGateway: { } I0414 05:13:08.155454 29748 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0414 05:13:08.155534 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.155563 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.155591 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.155603 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.155655 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.155679 29748 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0414 05:13:08.155703 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.155746 29748 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0414 05:13:08.155796 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0414 05:13:08.155881 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.155918 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.155949 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.155961 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.155985 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.156009 29748 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0414 05:13:08.156050 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0414 05:13:08.156623 29748 target.go:168] Cloudformation output is in /tmp/kops-render-3272798624 I0414 05:13:08.156820 29748 target.go:168] Cloudformation output is in /tmp/kops-render-3272798624 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0414 05:13:08.158167 29748 target.go:112] Terraform output is in /tmp/kops-render-1128483277 I0414 05:13:08.159660 29748 target.go:112] Terraform output is in /tmp/kops-render-1128483277 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0414 05:13:08.159988 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.160025 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.160094 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0414 05:13:08.160123 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.160139 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.160156 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.160205 29748 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0414 05:13:08.160337 29748 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0414 05:13:08.160368 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0414 05:13:08.160393 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0414 05:13:08.160487 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.160519 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.160551 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.160562 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.160581 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.160598 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.160648 29748 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0414 05:13:08.160712 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0414 05:13:08.160970 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.161001 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.161057 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0414 05:13:08.161083 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.161098 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.161116 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.161144 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.161215 29748 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0414 05:13:08.161242 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.161258 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.161275 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0414 05:13:08.161367 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.161397 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.161427 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.161438 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.161454 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.161471 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.161497 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.161533 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.161551 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0414 05:13:08.161726 29748 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0414 05:13:08.161757 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.161812 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0414 05:13:08.161836 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.161851 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.161867 29748 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0414 05:13:08.161922 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.162001 29748 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0414 05:13:08.162038 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.162055 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.162099 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.162303 29748 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:08.162362 29748 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0414 05:13:08.162568 29748 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0414 05:13:08.162640 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.162693 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.162711 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.162737 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.162760 29748 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0414 05:13:08.162806 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.162870 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.162895 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.162940 29748 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0414 05:13:08.163238 29748 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0414 05:13:08.163281 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.163365 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0414 05:13:08.163409 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.163433 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.163455 29748 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0414 05:13:08.163501 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.163581 29748 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0414 05:13:08.163606 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.163795 29748 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0414 05:13:08.164259 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.164329 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.164387 29748 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0414 05:13:08.164647 29748 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0414 05:13:08.164748 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.165063 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.165077 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.165100 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.165122 29748 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0414 05:13:08.165159 29748 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0414 05:13:08.165257 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.165277 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.165318 29748 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0414 05:13:08.165461 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0414 05:13:08.165516 29748 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0414 05:13:08.165629 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.165663 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.165693 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.165704 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.165755 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.165776 29748 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0414 05:13:08.165803 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.165830 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.165895 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0414 05:13:08.165914 29748 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0414 05:13:08.165937 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0414 05:13:08.166034 29748 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0414 05:13:08.166065 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.166095 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.166106 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.166129 29748 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0414 05:13:08.166155 29748 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0414 05:13:08.166194 29748 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0414 05:13:08.166367 29748 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0414 05:13:08.166399 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.166458 29748 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0414 05:13:08.166486 29748 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0414 05:13:08.166502 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.166519 29748 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0414 05:13:08.166592 29748 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0414 05:13:08.166623 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0414 05:13:08.166655 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.166666 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.166683 29748 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0414 05:13:08.166700 29748 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0414 05:13:08.166849 29748 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0414 05:13:08.166943 29748 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0414 05:13:08.166976 29748 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0414 05:13:08.167008 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0414 05:13:08.167019 29748 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0414 05:13:08.167041 29748 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.054s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.046s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.084s === RUN TestDiskRenderAzure I0414 05:13:04.135025 29107 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0414 05:13:04.135404 29107 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0414 05:13:04.135675 29107 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0414 05:13:04.136173 29107 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0414 05:13:04.136458 29107 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0414 05:13:04.136742 29107 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0414 05:13:04.136998 29107 resourcegroup.go:103] Creating a new Resource Group with name: rg I0414 05:13:04.137011 29107 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0414 05:13:04.137224 29107 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0414 05:13:04.137838 29107 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0414 05:13:04.138114 29107 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0414 05:13:04.138346 29107 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0414 05:13:04.138706 29107 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0414 05:13:04.138997 29107 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.066s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.086s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gcetasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.03s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.02s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.084s === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.060s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.026s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.031s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.028s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0414 05:13:09.567070 29788 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0414 05:13:09.567147 29788 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0414 05:13:09.567158 29788 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0414 05:13:09.567169 29788 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0414 05:13:09.567178 29788 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0414 05:13:09.567188 29788 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0414 05:13:09.567197 29788 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0414 05:13:09.567693 29788 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.056s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.081s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.048s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.026s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.071s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.044s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0414 05:13:05.386249 29461 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.019s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.050s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.040s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.059s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.020s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0414 05:13:10.397677 29916 s3context.go:89] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0414 05:13:10.397725 29916 s3context.go:237] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.056s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... kubernetes so:libc.musl-x86_64.so.1 >>> kops*: Package size: 156.6 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.23.0-r1.apk >>> kops: Build complete at Thu, 14 Apr 2022 05:14:05 +0000 elapsed time 0h 3m 10s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20220414.051056) (2/3) Purging kubernetes (1.23.5-r2) (3/3) Purging go (1.18.1-r1) Executing busybox-1.35.0-r8.trigger OK: 331 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...