>>> kops: Building testing/kops 1.23.0-r3 (using abuild 3.9.0-r2) started Thu, 21 Jul 2022 22:11:39 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.24.3-r1) (2/3) Installing go (1.18.4-r2) (3/3) Installing .makedepends-kops (20220721.221141) Executing busybox-1.35.0-r19.trigger OK: 713 MiB in 104 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 4 28.7M 4 1455k 0 0 8175k 0 0:00:03 --:--:-- 0:00:03 8179k 100 28.7M 100 28.7M 0 0 55.1M 0 --:--:-- --:--:-- --:--:-- 55.1M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.23.0.tar.gz >>> kops: Checking sha512sums... kops-1.23.0.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.23.0.tar.gz... go: downloading k8s.io/apimachinery v0.23.1 go: downloading github.com/aws/aws-sdk-go v1.43.11 go: downloading k8s.io/klog/v2 v2.30.0 go: downloading google.golang.org/api v0.57.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/gophercloud/gophercloud v0.24.0 go: downloading k8s.io/api v0.23.1 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading k8s.io/client-go v0.23.1 go: downloading github.com/blang/semver v3.5.1+incompatible go: downloading k8s.io/klog v1.0.0 go: downloading github.com/jetstack/cert-manager v1.6.1 go: downloading github.com/spf13/cobra v1.2.1 go: downloading github.com/spf13/viper v1.8.1 go: downloading k8s.io/cli-runtime v0.23.1 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading github.com/go-logr/logr v1.2.0 go: downloading github.com/prometheus/client_golang v1.11.0 go: downloading cloud.google.com/go v0.97.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/google/go-cmp v0.5.6 go: downloading github.com/digitalocean/godo v1.65.0 go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e go: downloading golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.23.1 go: downloading k8s.io/component-base v0.23.1 go: downloading sigs.k8s.io/controller-runtime v0.11.0 go: downloading github.com/google/go-containerregistry v0.7.0 go: downloading golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd go: downloading golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd go: downloading k8s.io/kubectl v0.23.1 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/stretchr/testify v1.7.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.0.2 go: downloading helm.sh/helm/v3 v3.7.1 go: downloading github.com/pelletier/go-toml v1.9.3 go: downloading k8s.io/kubelet v0.23.1 go: downloading k8s.io/mount-utils v0.23.1 go: downloading k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.13.0 go: downloading github.com/denverdino/aliyungo v0.0.0-20210425065611-55bee4942cba go: downloading github.com/go-ini/ini v1.62.0 go: downloading github.com/hashicorp/vault/api v1.1.1 go: downloading github.com/pkg/sftp v1.13.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading gopkg.in/square/go-jose.v2 v2.5.1 go: downloading github.com/fsnotify/fsnotify v1.5.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading github.com/magiconair/properties v1.8.5 go: downloading github.com/mitchellh/mapstructure v1.4.1 go: downloading github.com/spf13/afero v1.6.0 go: downloading github.com/spf13/cast v1.3.1 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/subosito/gotenv v1.2.0 go: downloading gopkg.in/ini.v1 v1.62.0 go: downloading github.com/inconshreveable/mousetrap v1.0.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/googleapis/gnostic v0.5.5 go: downloading github.com/Azure/azure-sdk-for-go v56.2.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest v0.11.19 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/prometheus/client_model v0.2.0 go: downloading github.com/prometheus/common v0.28.0 go: downloading github.com/google/go-querystring v1.0.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.27.1 go: downloading golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.16.1-0.20210702024009-ea6160c1d0e3 go: downloading k8s.io/cloud-provider v0.23.1 go: downloading github.com/docker/cli v20.10.10+incompatible go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.7 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/spotinst/spotinst-sdk-go v1.85.0 go: downloading github.com/aliyun/alibaba-cloud-sdk-go v1.61.1059 go: downloading github.com/google/go-tpm-tools v0.3.0-beta1 go: downloading github.com/google/go-tpm v0.3.2 go: downloading github.com/hashicorp/hcl/v2 v2.10.1 go: downloading github.com/zclconf/go-cty v1.8.2 go: downloading k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b go: downloading github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 go: downloading github.com/russross/blackfriday v1.5.2 go: downloading google.golang.org/grpc v1.42.0 go: downloading sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 go: downloading k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.1 go: downloading k8s.io/csi-translation-lib v0.23.1 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.14 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.0.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.1 go: downloading github.com/hashicorp/go-multierror v1.1.0 go: downloading github.com/hashicorp/go-retryablehttp v0.6.6 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/vault/sdk v0.2.1 go: downloading github.com/kr/fs v0.1.0 go: downloading k8s.io/apiextensions-apiserver v0.23.1 go: downloading golang.org/x/text v0.3.7 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.10.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.0 go: downloading github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.1.1 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.1 go: downloading github.com/prometheus/procfs v0.6.0 go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/docker/distribution v2.7.1+incompatible go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.2 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading go.uber.org/goleak v1.1.12 go: downloading github.com/onsi/gomega v1.17.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/smartystreets/goconvey v1.6.4 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/hashicorp/go-hclog v0.16.1 go: downloading github.com/go-logr/zapr v1.2.0 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.10.0 go: downloading go.uber.org/zap v1.19.1 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.1 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.1.4 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/moby/term v0.0.0-20210610120745-9d4ed1856297 go: downloading go.opencensus.io v0.23.0 go: downloading github.com/googleapis/gax-go/v2 v2.1.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/stretchr/objx v0.2.0 go: downloading github.com/ghodss/yaml v1.0.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/form3tech-oss/jwt-go v3.2.3+incompatible go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/mitchellh/reflectwalk v1.0.1 go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/google/go-attestation v0.3.2 go: downloading github.com/docker/docker v20.10.10+incompatible go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/opencontainers/image-spec v1.0.2-0.20210730191737-8e42a01fb1b7 go: downloading github.com/fatih/color v1.12.0 go: downloading github.com/mattn/go-isatty v0.0.12 go: downloading github.com/mattn/go-colorable v0.1.8 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading github.com/fvbommel/sortorder v1.0.1 go: downloading github.com/klauspost/compress v1.13.6 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading google.golang.org/genproto v0.0.0-20211111162719-482062a4217b go: downloading github.com/containerd/containerd v1.5.7 go: downloading github.com/sirupsen/logrus v1.8.1 go: downloading oras.land/oras-go v0.4.0 go: downloading gotest.tools v2.2.0+incompatible go: downloading github.com/distribution/distribution/v3 v3.0.0-20210804104954-38ab4c606ee3 go: downloading github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 go: downloading github.com/armon/go-metrics v0.3.3 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.35 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/golang/snappy v0.0.3 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading go.uber.org/atomic v1.7.0 go: downloading go.uber.org/multierr v1.6.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.6.1 go: downloading github.com/docker/docker-credential-helpers v0.6.4 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/smartystreets/assertions v1.1.0 go: downloading github.com/jtolds/gls v4.20.0+incompatible go: downloading github.com/google/certificate-transparency-go v1.1.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/hashicorp/go-immutable-radix v1.1.0 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v0.0.0-20181112141820-a009c3971eca go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading rsc.io/letsencrypt v0.0.3 go: downloading github.com/gopherjs/gopherjs v0.0.0-20200217142428-fce0ec30dd00 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading golang.org/x/mod v0.5.1 go: downloading github.com/Microsoft/hcsshim v0.8.22 go: downloading github.com/moby/sys/mountinfo v0.4.1 go: downloading github.com/Microsoft/go-winio v0.5.1 go: downloading github.com/containerd/cgroups v1.0.1 go: downloading github.com/docker/go-units v0.4.0 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading github.com/frankban/quicktest v1.11.3 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 go: downloading github.com/Masterminds/sprig v2.22.0+incompatible go: downloading github.com/cpuguy83/go-md2man v1.0.10 go: downloading github.com/Masterminds/semver v1.5.0 go: downloading github.com/cespare/xxhash v1.1.0 go: downloading github.com/apparentlymart/go-textseg v1.0.0 /bin/sh: tools/get_version.sh: not found go build -ldflags=all="-X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=v20220715-373-gedda498804 " -o /home/buildozer/aports/testing/kops/src/kops-1.23.0/.build/local/kops k8s.io/kops/cmd/kops/ go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0721 22:13:46.313999 56571 addon.go:218] addon test wants to update all nodes I0721 22:13:46.315908 56571 addon.go:218] addon test wants to update worker nodes I0721 22:13:46.316230 56571 addon.go:218] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0721 22:13:46.316635 56571 addon.go:251] installing PKI for "test" I0721 22:13:46.912838 56571 addon.go:251] installing PKI for "test" --- PASS: Test_InstallPKI (0.92s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.057s ? k8s.io/kops/channels/pkg/cmd [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I0721 22:14:05.357355 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.357433 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.357479 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.357532 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.357571 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.357595 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.357620 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.357651 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.357675 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.357728 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.357763 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.381788 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.381885 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.382269 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:05.403799 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.403848 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.403894 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.403942 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.403978 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.404008 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.404037 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.404065 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.404090 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.404143 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.404179 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.443720 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.443783 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.444023 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:05.450282 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.450313 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.450339 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.450366 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.450388 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.450407 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.450425 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.450442 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.450457 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.450492 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.450515 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.490970 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.491031 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.491220 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:05.496361 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.496386 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.496411 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.496437 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.496458 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.496476 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.496492 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.496507 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.496520 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.496549 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.496570 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.525127 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.525234 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.525816 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:05.531129 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.531160 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.531190 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.531219 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.531242 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.531260 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.531277 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.531295 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.531309 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.531344 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.531365 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.557724 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.557801 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.558021 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:05.563722 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.563752 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.563778 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.563807 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.563829 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.563846 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.563864 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.563882 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.563898 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.563934 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.563957 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.586538 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.586602 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.586793 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.24s) === RUN TestCreateClusterOverride I0721 22:14:05.592485 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.592511 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.592536 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.592562 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.592585 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.592602 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.592620 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.592636 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.592650 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.592706 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.592730 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.658094 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.658758 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.659208 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.08s) === RUN TestCreateClusterComplex I0721 22:14:05.669247 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.669302 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.669352 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.669403 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.669447 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.669476 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.669507 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.669538 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.669564 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.669660 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.669714 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.727342 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.727417 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.727669 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.06s) === RUN TestCreateClusterHA I0721 22:14:05.733277 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.733308 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.733334 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.733364 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.733390 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.733426 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.733450 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.733470 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.733485 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.733535 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.733559 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.784469 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.784554 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.784573 59970 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0721 22:14:05.784584 59970 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0721 22:14:05.784897 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:05.797217 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.797260 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.797289 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.797318 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.797344 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.797361 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.797379 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.797398 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.797415 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.797458 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.797481 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.831718 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:05.831789 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:05.831804 59970 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0721 22:14:05.831814 59970 subnets.go:182] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0721 22:14:05.832077 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.11s) === RUN TestCreateClusterHAGCE I0721 22:14:05.841131 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.841157 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.841181 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.841206 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.841229 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.841245 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.841262 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.841277 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.841291 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.841336 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.841358 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0721 22:14:05.865252 59970 new_cluster.go:372] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0721 22:14:05.865271 59970 new_cluster.go:373] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0721 22:14:05.865323 59970 new_cluster.go:1102] Cloud Provider ID = gce I0721 22:14:05.865409 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.04s) === RUN TestCreateClusterGCE I0721 22:14:05.877753 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:05.877816 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:05.877862 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:05.877909 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:05.877947 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:05.877976 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:05.878003 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:05.878030 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:05.878059 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:05.878132 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:05.878173 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.063675 59970 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0721 22:14:06.063721 59970 new_cluster.go:1102] Cloud Provider ID = gce I0721 22:14:06.063747 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.19s) === RUN TestCreateClusterHASharedZone I0721 22:14:06.069178 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.069205 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.069228 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.069252 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.069273 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.069289 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.069307 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.069324 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.069337 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.069375 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.069398 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0721 22:14:06.115627 59970 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0721 22:14:06.115716 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.115837 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:06.116276 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.06s) === RUN TestCreateClusterHASharedZones I0721 22:14:06.128772 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.128821 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.128866 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.128915 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.128953 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.128981 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.129009 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.129037 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.129064 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.129135 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.129174 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0721 22:14:06.166403 59970 new_cluster.go:750] Running with masters in the same AZs; redundancy will be reduced I0721 22:14:06.166464 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.166534 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:06.166548 59970 subnets.go:182] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0721 22:14:06.166845 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.05s) === RUN TestCreateClusterPrivate I0721 22:14:06.177152 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.177185 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.177210 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.177240 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.177267 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.177283 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.177301 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.177322 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.177339 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.177387 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.177413 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.266254 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.266343 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:06.266361 59970 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0721 22:14:06.266601 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:06.266630 59970 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.10s) === RUN TestCreateClusterPrivateGCE I0721 22:14:06.273326 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.273351 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.273379 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.273408 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.273431 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.273450 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.273468 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.273484 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.273498 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.273539 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.273562 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.399162 59970 new_cluster.go:369] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0721 22:14:06.399210 59970 new_cluster.go:1102] Cloud Provider ID = gce I0721 22:14:06.399251 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.14s) === RUN TestCreateClusterWithNGWSpecified I0721 22:14:06.412929 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.412980 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.413030 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.413065 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.413108 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.413144 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.413182 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.413208 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.413226 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.413283 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.413312 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.498688 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.498746 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:06.498759 59970 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0721 22:14:06.498959 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:06.498987 59970 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.09s) === RUN TestCreateClusterWithINGWSpecified I0721 22:14:06.505015 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.505042 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.505064 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.505086 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.505106 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.505122 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.505140 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.505156 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.505170 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.505207 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.505230 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.541404 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.541455 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:06.541468 59970 subnets.go:196] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0721 22:14:06.541655 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:06.541682 59970 utils.go:203] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.05s) === RUN TestCreateClusterSharedVPC I0721 22:14:06.553274 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.553318 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.553361 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.553424 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.553463 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.553490 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.553519 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.553550 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.553576 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.553643 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.553681 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.652015 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.652055 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.652098 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.652127 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.652141 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.652174 59970 subnets.go:182] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0721 22:14:06.652359 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.10s) === RUN TestCreateClusterSharedSubnets I0721 22:14:06.657493 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.657520 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.657542 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.657568 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.657589 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.657605 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.657621 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.657635 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.657648 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.657690 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.657714 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.725166 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.725200 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.725257 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.725294 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.725316 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.725341 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.725358 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.725574 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.08s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0721 22:14:06.733039 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.733070 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.733101 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.733132 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.733158 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.733179 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.733269 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.733292 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.733307 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.733356 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.733387 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.801498 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:06.801552 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.801574 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.801642 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.801689 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.801709 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.801732 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.801747 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.802037 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.07s) === RUN TestCreateClusterPrivateSharedSubnets I0721 22:14:06.807737 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.807769 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.807806 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.807837 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.807862 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.807879 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.807897 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.807915 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.807932 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.807982 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.808005 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.895173 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.895223 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.895300 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.895327 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.895354 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.895403 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.895421 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.895444 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:06.895459 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:06.895796 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.09s) === RUN TestCreateClusterIPv6 I0721 22:14:06.901917 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.901952 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.901977 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.902008 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.902035 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.902051 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.902069 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.902091 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.902106 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.902155 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.902179 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0721 22:14:06.948650 59970 new_cluster.go:1102] Cloud Provider ID = aws I0721 22:14:06.948734 59970 subnets.go:182] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0721 22:14:06.949092 59970 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.05s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestMinimal I0721 22:14:06.955533 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:06.955579 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:06.955610 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:06.955643 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:06.955702 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:06.955719 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:06.955739 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:06.955768 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:06.955785 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:06.971623 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:06.973270 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:06.973351 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0721 22:14:09.867131 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:12.378192 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:12.379058 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:12.382864 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:12.382978 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:12.383219 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:12.383242 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:12.383499 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:12.384923 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:12.388009 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:12.388510 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.388880 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.389860 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.390395 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.390517 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:12.391221 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:12.443947 59970 target.go:112] Terraform output is in /tmp/test1304965785/out I0721 22:14:12.448096 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:12.448159 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:12.448214 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:12.448268 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:12.448311 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:12.448341 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:12.448372 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:12.448408 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:12.448435 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:12.473929 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:12.476942 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.477041 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0721 22:14:12.520374 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:12.617312 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0721 22:14:12.617870 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:12.617970 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0721 22:14:12.618100 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:12.618029 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:12.618151 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:12.618201 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:12.618284 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:12.618298 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:12.618381 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:12.619899 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:12.627300 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:12.627853 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.628128 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.629187 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.630073 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.630203 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:12.630465 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:12.635387 59970 target.go:168] Cloudformation output is in /tmp/test2562589612/out --- PASS: TestMinimal (5.69s) === RUN TestMinimal_v1_23 I0721 22:14:12.646475 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:12.646526 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:12.646570 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:12.646621 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:12.646702 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:12.646733 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:12.646767 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:12.646801 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:12.646827 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:12.676511 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* I0721 22:14:12.679503 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:12.679605 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.472325 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:14.638026 59970 executor.go:111] Tasks: 0 done / 91 total; 46 can run W0721 22:14:14.638330 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:14.638681 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:14.638987 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:14.639118 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:14.639030 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:14.639202 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:14.641354 59970 executor.go:111] Tasks: 46 done / 91 total; 19 can run I0721 22:14:14.650383 59970 executor.go:111] Tasks: 65 done / 91 total; 24 can run I0721 22:14:14.650828 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.651050 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.652404 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.653580 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.653706 59970 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0721 22:14:14.654083 59970 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0721 22:14:14.711355 59970 target.go:112] Terraform output is in /tmp/test3004750331/out I0721 22:14:14.715783 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:14.715844 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:14.715896 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:14.715949 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:14.715992 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:14.716020 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:14.716055 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:14.716090 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:14.716115 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:14.739001 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:14.741801 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.741899 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0721 22:14:14.782756 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:14.886702 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0721 22:14:14.888339 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:14.888845 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:14.888837 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:14.888974 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:14.888893 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:14.889057 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:14.889065 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:14.889117 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:14.891391 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:14.893762 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:14.899421 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:14.899860 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.900133 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.901560 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.903629 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.903896 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:14.904462 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:14.909630 59970 target.go:168] Cloudformation output is in /tmp/test1664893363/out --- PASS: TestMinimal_v1_23 (2.27s) === RUN TestNvidia I0721 22:14:14.919901 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:14.919953 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:14.920002 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:14.920056 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:14.920143 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:14.920175 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:14.920207 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:14.920242 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:14.920269 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:14.944579 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:14.947213 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.947301 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:14.986109 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:15.103768 59970 executor.go:111] Tasks: 0 done / 87 total; 45 can run I0721 22:14:15.104176 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:15.104299 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:15.104330 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:15.104649 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:15.104721 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:15.104845 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:15.106693 59970 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0721 22:14:15.115148 59970 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0721 22:14:15.115661 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.115901 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.117057 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.118029 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.118151 59970 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0721 22:14:15.118493 59970 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0721 22:14:15.177069 59970 target.go:112] Terraform output is in /tmp/test1884665926/out I0721 22:14:15.181454 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:15.181516 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:15.181566 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:15.181619 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:15.181662 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:15.181692 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:15.181722 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:15.181756 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:15.181785 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:15.205538 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:15.207138 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.207195 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.245800 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:15.356171 59970 executor.go:111] Tasks: 0 done / 87 total; 45 can run W0721 22:14:15.356494 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:15.356623 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:15.356664 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:15.356759 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:15.356944 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:15.357069 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:15.357256 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:15.357287 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:15.357454 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I0721 22:14:15.359652 59970 executor.go:111] Tasks: 45 done / 87 total; 19 can run I0721 22:14:15.364917 59970 executor.go:111] Tasks: 64 done / 87 total; 21 can run I0721 22:14:15.365295 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.365608 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.366789 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.367631 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.367767 59970 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0721 22:14:15.368059 59970 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0721 22:14:15.371575 59970 target.go:168] Cloudformation output is in /tmp/test261720491/out --- PASS: TestNvidia (0.46s) === RUN TestMinimalGossip I0721 22:14:15.384304 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:15.384363 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:15.384414 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:15.384483 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:15.384530 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:15.384561 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:15.384591 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:15.384627 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:15.384654 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:15.411764 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.411863 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.451094 59970 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0721 22:14:15.556162 59970 executor.go:111] Tasks: 0 done / 85 total; 43 can run I0721 22:14:15.556662 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:15.556888 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:15.557292 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:15.563542 59970 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0721 22:14:15.569493 59970 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0721 22:14:15.569781 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.570147 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.571330 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.572172 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.572294 59970 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0721 22:14:15.572675 59970 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0721 22:14:15.626684 59970 target.go:112] Terraform output is in /tmp/test1136725559/out --- PASS: TestMinimalGossip (0.25s) === RUN TestMinimalGossipIRSA I0721 22:14:15.636722 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:15.636787 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:15.636839 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:15.636890 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:15.636933 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:15.636961 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:15.636990 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:15.637021 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:15.637048 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:15.665742 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.665838 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.704789 59970 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0721 22:14:15.816284 59970 executor.go:111] Tasks: 0 done / 88 total; 45 can run W0721 22:14:15.816585 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:15.816681 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:15.818107 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:15.822427 59970 executor.go:111] Tasks: 45 done / 88 total; 20 can run I0721 22:14:15.826056 59970 executor.go:111] Tasks: 65 done / 88 total; 21 can run I0721 22:14:15.826470 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.826687 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.828953 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.831676 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:15.831902 59970 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0721 22:14:15.832214 59970 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0721 22:14:15.875638 59970 target.go:112] Terraform output is in /tmp/test1674277627/out --- PASS: TestMinimalGossipIRSA (0.25s) === RUN TestMinimalGCE I0721 22:14:15.879921 59970 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0721 22:14:15.881479 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0721 22:14:16.786131 59970 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0721 22:14:16.786228 59970 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0721 22:14:16.786406 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0721 22:14:16.786458 59970 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0721 22:14:16.786482 59970 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0721 22:14:16.786513 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:16.786615 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:16.786644 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:16.788308 59970 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0721 22:14:16.791728 59970 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0721 22:14:16.801440 59970 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0721 22:14:16.802977 59970 instancetemplate.go:555] adding csa: default I0721 22:14:16.804151 59970 instancetemplate.go:555] adding csa: default I0721 22:14:16.804275 59970 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0721 22:14:16.804758 59970 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0721 22:14:16.836493 59970 target.go:112] Terraform output is in /tmp/test1423953434/out --- PASS: TestMinimalGCE (0.96s) === RUN TestMinimalGCEPrivate I0721 22:14:16.840541 59970 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0721 22:14:16.842402 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0721 22:14:17.653467 59970 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0721 22:14:17.653559 59970 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0721 22:14:17.653704 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0721 22:14:17.653748 59970 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0721 22:14:17.653772 59970 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0721 22:14:17.653792 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:17.653864 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:17.653888 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:17.655608 59970 executor.go:111] Tasks: 0 done / 63 total; 37 can run I0721 22:14:17.663243 59970 executor.go:111] Tasks: 37 done / 63 total; 18 can run I0721 22:14:17.668867 59970 executor.go:111] Tasks: 55 done / 63 total; 4 can run I0721 22:14:17.671549 59970 instancetemplate.go:555] adding csa: default I0721 22:14:17.671694 59970 executor.go:111] Tasks: 59 done / 63 total; 3 can run I0721 22:14:17.673273 59970 instancetemplate.go:555] adding csa: default I0721 22:14:17.673387 59970 executor.go:111] Tasks: 62 done / 63 total; 1 can run I0721 22:14:17.673539 59970 executor.go:111] Tasks: 63 done / 63 total; 0 can run I0721 22:14:17.704523 59970 target.go:112] Terraform output is in /tmp/test2871802869/out --- PASS: TestMinimalGCEPrivate (0.87s) === RUN TestHA I0721 22:14:17.708975 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:17.709033 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:17.709086 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:17.709140 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:17.709185 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:17.709213 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:17.709243 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:17.709276 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:17.709302 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:17.740444 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:17.743020 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.743101 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.743153 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.743201 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.783587 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:17.896107 59970 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0721 22:14:17.896670 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:17.896766 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:17.896800 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:17.896894 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:17.896856 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:17.896930 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:17.897094 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:17.897239 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:17.899165 59970 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0721 22:14:17.908573 59970 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0721 22:14:17.909112 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.909349 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.909626 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.909949 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.911151 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.911778 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.911997 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.912156 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:17.912285 59970 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0721 22:14:17.912898 59970 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0721 22:14:17.991959 59970 target.go:112] Terraform output is in /tmp/test1257055318/out --- PASS: TestHA (0.29s) === RUN TestHighAvailabilityGCE I0721 22:14:17.996722 59970 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true I0721 22:14:17.998806 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* W0721 22:14:18.193475 59970 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0721 22:14:18.193566 59970 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0721 22:14:18.193704 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0721 22:14:18.193747 59970 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0721 22:14:18.193770 59970 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0721 22:14:18.193790 59970 autoscalinggroup.go:168] gsa: default W0721 22:14:18.193850 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0721 22:14:18.193884 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:18.193905 59970 autoscalinggroup.go:168] gsa: default W0721 22:14:18.193957 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups I0721 22:14:18.193989 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:18.194012 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:18.194074 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:18.194098 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:18.196179 59970 executor.go:111] Tasks: 0 done / 76 total; 41 can run I0721 22:14:18.199496 59970 executor.go:111] Tasks: 41 done / 76 total; 20 can run I0721 22:14:18.209106 59970 executor.go:111] Tasks: 61 done / 76 total; 7 can run I0721 22:14:18.211905 59970 instancetemplate.go:555] adding csa: default I0721 22:14:18.213921 59970 instancetemplate.go:555] adding csa: default I0721 22:14:18.213933 59970 instancetemplate.go:555] adding csa: default I0721 22:14:18.214159 59970 executor.go:111] Tasks: 68 done / 76 total; 5 can run I0721 22:14:18.215421 59970 instancetemplate.go:555] adding csa: default I0721 22:14:18.215556 59970 executor.go:111] Tasks: 73 done / 76 total; 3 can run I0721 22:14:18.215886 59970 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0721 22:14:18.263399 59970 target.go:112] Terraform output is in /tmp/test2958276558/out --- PASS: TestHighAvailabilityGCE (0.27s) === RUN TestComplex I0721 22:14:18.268547 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:18.268605 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:18.268654 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:18.268707 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:18.268794 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:18.268823 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:18.268852 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:18.268886 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:18.268913 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:18.300561 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:18.304131 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.304230 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.343812 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:18.450412 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0721 22:14:18.450977 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:18.451466 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:18.451512 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:18.454232 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:18.454421 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:18.457804 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:18.463434 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:18.463738 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.464172 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.464862 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.466859 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.466981 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:18.467698 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0721 22:14:18.542803 59970 target.go:112] Terraform output is in /tmp/test3055830035/out I0721 22:14:18.546816 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:18.546879 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:18.546929 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:18.546980 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:18.547026 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:18.547057 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:18.547089 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:18.547123 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:18.547149 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:18.578782 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:18.582552 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.582646 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.621429 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:18.724742 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0721 22:14:18.725301 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:18.725300 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:18.725347 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:18.725435 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:18.725443 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:18.732342 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:18.738479 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:18.739232 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.739471 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.740360 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.742112 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.742257 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:18.742599 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0721 22:14:18.750619 59970 target.go:168] Cloudformation output is in /tmp/test1698493172/out I0721 22:14:18.764177 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:18.764228 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:18.764275 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:18.764324 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:18.764364 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:18.764392 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:18.764423 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:18.764471 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:18.764498 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:18.794689 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:18.797645 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.797724 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.838118 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:18.948840 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run W0721 22:14:18.949372 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:18.949792 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:18.949927 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:18.949970 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:18.950117 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:18.952423 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:18.957746 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:18.958106 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.959225 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.963349 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.964105 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:18.964248 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:18.964713 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0721 22:14:19.036771 59970 target.go:112] Terraform output is in /tmp/test3721788126/out --- PASS: TestComplex (0.78s) === RUN TestCompress I0721 22:14:19.043695 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:19.043754 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:19.043807 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:19.043860 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:19.043903 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:19.043933 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:19.043963 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:19.043997 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:19.044025 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:19.069397 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:19.072280 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.072390 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.111099 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:19.217493 59970 executor.go:111] Tasks: 0 done / 85 total; 43 can run W0721 22:14:19.218229 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:19.218578 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:19.218620 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:19.218621 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:19.218747 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:19.224506 59970 executor.go:111] Tasks: 43 done / 85 total; 19 can run I0721 22:14:19.230311 59970 executor.go:111] Tasks: 62 done / 85 total; 21 can run I0721 22:14:19.230748 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.230982 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.235696 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.238390 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.238529 59970 executor.go:111] Tasks: 83 done / 85 total; 2 can run I0721 22:14:19.238885 59970 executor.go:111] Tasks: 85 done / 85 total; 0 can run I0721 22:14:19.287101 59970 target.go:112] Terraform output is in /tmp/test2222882251/out --- PASS: TestCompress (0.25s) === RUN TestExternalPolicies I0721 22:14:19.291383 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:19.291440 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:19.291490 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:19.291541 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:19.291584 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:19.291615 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:19.291644 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:19.291678 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:19.291703 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:19.324056 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:19.327396 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.327505 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.367735 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:19.473808 59970 executor.go:111] Tasks: 0 done / 99 total; 46 can run I0721 22:14:19.474680 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:19.474757 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:19.474721 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:19.474836 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:19.474861 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:19.474990 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:19.479211 59970 executor.go:111] Tasks: 46 done / 99 total; 20 can run I0721 22:14:19.486225 59970 executor.go:111] Tasks: 66 done / 99 total; 30 can run I0721 22:14:19.486744 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.486978 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.488140 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.489232 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.489362 59970 executor.go:111] Tasks: 96 done / 99 total; 3 can run I0721 22:14:19.489790 59970 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0721 22:14:19.557743 59970 target.go:112] Terraform output is in /tmp/test1124368838/out --- PASS: TestExternalPolicies (0.27s) === RUN TestMinimalIPv6 I0721 22:14:19.562196 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:19.562261 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:19.562313 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:19.562369 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:19.562465 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:19.562495 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:19.562526 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:19.562562 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:19.562590 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:19.587881 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:19.591187 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.591287 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.630112 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:19.804182 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0721 22:14:19.804986 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:19.805241 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:19.805264 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:19.805293 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:19.805381 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:19.805404 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:19.807659 59970 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0721 22:14:19.814161 59970 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0721 22:14:19.814805 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.815101 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.816839 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.818544 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.818712 59970 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0721 22:14:19.819197 59970 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0721 22:14:19.819502 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:19.888766 59970 target.go:112] Terraform output is in /tmp/test4041985162/out I0721 22:14:19.893338 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:19.893403 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:19.893455 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:19.893506 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:19.893553 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:19.893581 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:19.893610 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:19.893645 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:19.893672 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:19.918735 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:19.921992 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.922087 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:19.962485 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:20.134452 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0721 22:14:20.134874 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:20.135173 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:20.135349 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:20.135380 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:20.135965 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:20.137051 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:20.137235 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:20.137260 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:20.137519 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:20.142110 59970 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0721 22:14:20.147310 59970 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0721 22:14:20.147716 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.148057 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.149253 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.150470 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.150641 59970 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0721 22:14:20.151093 59970 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0721 22:14:20.151497 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:20.155903 59970 target.go:168] Cloudformation output is in /tmp/test1726559901/out --- PASS: TestMinimalIPv6 (0.61s) === RUN TestMinimalIPv6Private I0721 22:14:20.167771 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:20.167859 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:20.167911 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:20.167965 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:20.168047 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:20.168076 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:20.168108 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:20.168142 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:20.168170 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:20.200209 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:20.203596 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.203694 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.244044 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:20.423120 59970 executor.go:111] Tasks: 0 done / 118 total; 46 can run W0721 22:14:20.423528 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:20.423656 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:20.424214 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:20.424251 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:20.424390 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:20.424930 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:20.426451 59970 executor.go:111] Tasks: 46 done / 118 total; 23 can run I0721 22:14:20.436185 59970 executor.go:111] Tasks: 69 done / 118 total; 34 can run I0721 22:14:20.436674 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.436889 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.438103 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.439384 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.439509 59970 executor.go:111] Tasks: 103 done / 118 total; 9 can run I0721 22:14:20.440124 59970 executor.go:111] Tasks: 112 done / 118 total; 6 can run I0721 22:14:20.440619 59970 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0721 22:14:20.512454 59970 target.go:112] Terraform output is in /tmp/test2170834652/out --- PASS: TestMinimalIPv6Private (0.35s) === RUN TestMinimalIPv6Calico I0721 22:14:20.518526 59970 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0721 22:14:20.522729 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:20.522799 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:20.522850 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:20.522899 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:20.522983 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:20.523016 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:20.523053 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:20.523089 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:20.523118 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:20.548100 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:20.551803 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.551905 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:20.592103 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:21.003290 59970 executor.go:111] Tasks: 0 done / 107 total; 47 can run W0721 22:14:21.003824 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:21.004104 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:21.004623 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:21.004936 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:21.006933 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:21.007243 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:21.011922 59970 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0721 22:14:21.017353 59970 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0721 22:14:21.017845 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.018042 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.019255 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.024742 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.024895 59970 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0721 22:14:21.025444 59970 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0721 22:14:21.025743 59970 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0721 22:14:21.089859 59970 target.go:112] Terraform output is in /tmp/test3696783215/out I0721 22:14:21.095281 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:21.095344 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:21.095400 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:21.095455 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:21.095498 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:21.095526 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:21.095555 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:21.095588 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:21.095617 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:21.125183 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:21.128423 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.128523 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.166819 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:21.547071 59970 executor.go:111] Tasks: 0 done / 107 total; 47 can run I0721 22:14:21.547435 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0721 22:14:21.547537 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:21.547470 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:21.547587 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:21.550201 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:21.550440 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:21.550471 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:21.550600 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:21.550609 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:21.555662 59970 executor.go:111] Tasks: 47 done / 107 total; 21 can run I0721 22:14:21.561101 59970 executor.go:111] Tasks: 68 done / 107 total; 31 can run I0721 22:14:21.561525 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.561652 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.563025 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.564661 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.564817 59970 executor.go:111] Tasks: 99 done / 107 total; 5 can run I0721 22:14:21.565440 59970 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0721 22:14:21.565739 59970 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0721 22:14:21.569380 59970 target.go:168] Cloudformation output is in /tmp/test2648319068/out I0721 22:14:21.584483 59970 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Calico (1.07s) === RUN TestMinimalIPv6Cilium I0721 22:14:21.584603 59970 featureflag.go:162] FeatureFlag "AWSIPv6"=true I0721 22:14:21.584841 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:21.584892 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:21.584940 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:21.585007 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:21.585158 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:21.585216 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:21.585272 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:21.585334 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:21.585380 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:21.610930 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:21.614336 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.614426 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.653569 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:21.875661 59970 executor.go:111] Tasks: 0 done / 106 total; 47 can run W0721 22:14:21.876287 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:21.876553 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:21.876610 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:21.876647 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:21.876733 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:21.876794 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:21.880212 59970 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0721 22:14:21.887540 59970 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0721 22:14:21.888062 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.888407 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.889303 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.891519 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.891749 59970 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0721 22:14:21.892150 59970 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0721 22:14:21.892455 59970 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0721 22:14:21.960579 59970 target.go:112] Terraform output is in /tmp/test148992157/out I0721 22:14:21.967291 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:21.967359 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:21.967417 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:21.967475 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:21.967519 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:21.967547 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:21.967578 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:21.967611 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:21.967637 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:21.994251 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:21.997809 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:21.997921 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.039121 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:22.259408 59970 executor.go:111] Tasks: 0 done / 106 total; 47 can run I0721 22:14:22.260056 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:22.260136 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:22.260186 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:22.260256 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:22.260202 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:22.260279 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:22.260368 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:22.260297 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } W0721 22:14:22.260382 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:22.262341 59970 executor.go:111] Tasks: 47 done / 106 total; 21 can run I0721 22:14:22.271381 59970 executor.go:111] Tasks: 68 done / 106 total; 30 can run I0721 22:14:22.271880 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.272143 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.273170 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.275143 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.275307 59970 executor.go:111] Tasks: 98 done / 106 total; 5 can run I0721 22:14:22.275784 59970 executor.go:111] Tasks: 103 done / 106 total; 3 can run I0721 22:14:22.276105 59970 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0721 22:14:22.280441 59970 target.go:168] Cloudformation output is in /tmp/test723680067/out I0721 22:14:22.291511 59970 featureflag.go:162] FeatureFlag "AWSIPv6"=false --- PASS: TestMinimalIPv6Cilium (0.71s) === RUN TestMinimalWarmPool I0721 22:14:22.291857 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:22.291912 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:22.291955 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:22.292005 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:22.292093 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:22.292122 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:22.292151 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:22.292186 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:22.292214 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:22.317050 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:22.323129 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.323231 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.363117 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:22.568976 59970 executor.go:111] Tasks: 0 done / 89 total; 46 can run W0721 22:14:22.569320 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:22.569437 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:22.570792 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:22.570877 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:22.571012 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:22.572441 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:22.576822 59970 executor.go:111] Tasks: 46 done / 89 total; 19 can run I0721 22:14:22.582103 59970 executor.go:111] Tasks: 65 done / 89 total; 21 can run I0721 22:14:22.582413 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.582605 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.583514 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.585183 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.585298 59970 executor.go:111] Tasks: 86 done / 89 total; 2 can run I0721 22:14:22.585596 59970 executor.go:111] Tasks: 88 done / 89 total; 1 can run I0721 22:14:22.586021 59970 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0721 22:14:22.641778 59970 target.go:112] Terraform output is in /tmp/test706639924/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I0721 22:14:22.646529 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:22.646591 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:22.646643 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:22.646696 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:22.646791 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:22.646820 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:22.646850 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:22.646886 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:22.646914 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:22.672819 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:22.676367 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.676464 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.716081 59970 zones.go:93] ListHostedZonesPages { } W0721 22:14:22.805626 59970 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0721 22:14:22.805682 59970 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 10 W0721 22:14:22.807500 59970 model.go:247] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0721 22:14:22.807647 59970 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_HOURLY_BACKUPS_RETENTION=1d W0721 22:14:22.807683 59970 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_MANAGER_DAILY_BACKUPS_RETENTION=30d ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:22.820066 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:22.820623 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:22.820659 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:22.820759 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:22.820838 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:22.822436 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:22.822660 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:22.822694 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:22.822849 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-etcd.example.com" },{ Key: "KubernetesCluster", Value: "minimal-etcd.example.com" },{ Key: "kubernetes.io/cluster/minimal-etcd.example.com", Value: "owned" }] }] } W0721 22:14:22.820856 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:22.827239 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:22.833156 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:22.833582 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.833910 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.834881 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.836663 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.836827 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:22.837128 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:22.840127 59970 target.go:168] Cloudformation output is in /tmp/test2136687867/out --- PASS: TestMinimalEtcd (0.20s) === RUN TestMinimalGp3 I0721 22:14:22.850909 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:22.850970 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:22.851039 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:22.851144 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:22.851267 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:22.851344 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:22.851403 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:22.851475 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:22.851518 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:22.878475 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:22.881297 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.881392 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:22.919696 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:23.031668 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0721 22:14:23.032197 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:23.032408 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:23.032794 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:23.032810 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:23.032834 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:23.032944 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:23.034944 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:23.040214 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:23.040430 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.040837 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.043237 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.046655 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.046794 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:23.047098 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:23.098373 59970 target.go:112] Terraform output is in /tmp/test1444381701/out I0721 22:14:23.105234 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:23.105314 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:23.105381 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:23.105452 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:23.105506 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:23.105540 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:23.105573 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:23.105623 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:23.105650 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:23.131181 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:23.134014 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.134108 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.175489 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:23.269320 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:23.269848 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:23.269906 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:23.269955 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0721 22:14:23.269974 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key W0721 22:14:23.270037 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:23.269981 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:23.270074 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:23.270012 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:23.270120 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0721 22:14:23.272810 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:23.278694 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:23.278881 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.279368 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.279570 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.281795 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.281927 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:23.282156 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:23.283960 59970 target.go:168] Cloudformation output is in /tmp/test1573066179/out --- PASS: TestMinimalGp3 (0.44s) === RUN TestExistingIAMCloudformation I0721 22:14:23.290053 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:23.290098 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:23.290128 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:23.290161 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:23.290215 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:23.290234 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:23.290253 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:23.290275 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:23.290292 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:23.313674 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:23.316516 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.316610 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.355088 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:23.452004 59970 executor.go:111] Tasks: 0 done / 76 total; 42 can run W0721 22:14:23.452546 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:23.452547 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:23.454642 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:23.454715 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:23.454689 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:23.454815 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:23.454957 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:23.454985 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:23.455141 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:23.458882 59970 executor.go:111] Tasks: 42 done / 76 total; 11 can run I0721 22:14:23.463742 59970 executor.go:111] Tasks: 53 done / 76 total; 21 can run I0721 22:14:23.464079 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.464237 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.465108 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.466935 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.467076 59970 executor.go:111] Tasks: 74 done / 76 total; 2 can run I0721 22:14:23.467283 59970 executor.go:111] Tasks: 76 done / 76 total; 0 can run I0721 22:14:23.469757 59970 target.go:168] Cloudformation output is in /tmp/test3363511981/out --- PASS: TestExistingIAMCloudformation (0.19s) === RUN TestExistingSG I0721 22:14:23.478062 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:23.478113 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:23.478161 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:23.478209 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:23.478261 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:23.478291 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:23.478321 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:23.478359 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:23.478389 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:23.516099 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:23.519208 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.519305 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.519367 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.519425 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.558940 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:23.663942 59970 executor.go:111] Tasks: 0 done / 139 total; 50 can run I0721 22:14:23.664408 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:23.664609 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:23.665129 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:23.665296 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:23.665815 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:23.665860 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:23.665992 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:23.666747 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:23.670827 59970 executor.go:111] Tasks: 50 done / 139 total; 26 can run I0721 22:14:23.676376 59970 executor.go:111] Tasks: 76 done / 139 total; 58 can run I0721 22:14:23.677080 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.677384 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.678251 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.678542 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.679323 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.682952 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.683151 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.683624 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.683764 59970 executor.go:111] Tasks: 134 done / 139 total; 5 can run I0721 22:14:23.684226 59970 executor.go:111] Tasks: 139 done / 139 total; 0 can run I0721 22:14:23.774477 59970 target.go:112] Terraform output is in /tmp/test2573836533/out --- PASS: TestExistingSG (0.30s) === RUN TestBastionAdditionalUserData I0721 22:14:23.780352 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:23.780404 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:23.780442 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:23.780476 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:23.780505 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:23.780522 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:23.780542 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:23.780565 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:23.780584 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:23.798160 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:23.800344 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.800409 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.800451 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.838152 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:23.928969 59970 executor.go:111] Tasks: 0 done / 120 total; 49 can run W0721 22:14:23.929428 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:23.929562 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:23.929721 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:23.929965 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:23.929986 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:23.930058 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:23.930053 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:23.932060 59970 executor.go:111] Tasks: 49 done / 120 total; 29 can run I0721 22:14:23.938072 59970 executor.go:111] Tasks: 78 done / 120 total; 36 can run I0721 22:14:23.939002 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.939159 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.939303 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.939514 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.939993 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.941387 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:23.941507 59970 executor.go:111] Tasks: 114 done / 120 total; 5 can run I0721 22:14:23.942409 59970 executor.go:111] Tasks: 119 done / 120 total; 1 can run I0721 22:14:23.942720 59970 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0721 22:14:24.013434 59970 target.go:112] Terraform output is in /tmp/test661670400/out --- PASS: TestBastionAdditionalUserData (0.24s) === RUN TestPrivateWeave I0721 22:14:24.016449 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:24.016496 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:24.016529 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:24.016563 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:24.016624 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:24.016642 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:24.016660 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:24.016682 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:24.016699 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:24.032957 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:24.034757 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.034819 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.034854 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.073364 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:24.170563 59970 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0721 22:14:24.170872 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:24.170987 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:24.171411 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:24.171448 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:24.172661 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:24.172940 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:24.172300 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:24.177949 59970 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0721 22:14:24.183921 59970 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0721 22:14:24.184494 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.184679 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.185020 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.185150 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.187403 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.191279 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.191444 59970 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0721 22:14:24.191915 59970 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0721 22:14:24.192032 59970 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0721 22:14:24.254431 59970 target.go:112] Terraform output is in /tmp/test1591900470/out --- PASS: TestPrivateWeave (0.24s) === RUN TestPrivateFlannel I0721 22:14:24.259408 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:24.259467 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:24.259524 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:24.259577 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:24.259670 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:24.259700 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:24.259732 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:24.259765 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:24.259792 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:24.287678 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:24.290718 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.290809 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.290875 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.331575 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:24.454613 59970 executor.go:111] Tasks: 0 done / 118 total; 48 can run I0721 22:14:24.455022 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:24.455078 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:24.455209 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:24.455272 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:24.455535 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:24.455607 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:24.455839 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:24.458013 59970 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0721 22:14:24.463603 59970 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0721 22:14:24.463942 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.464303 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.464474 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.464645 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.465185 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.466656 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.466784 59970 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0721 22:14:24.467195 59970 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0721 22:14:24.467362 59970 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0721 22:14:24.549271 59970 target.go:112] Terraform output is in /tmp/test587669308/out --- PASS: TestPrivateFlannel (0.29s) === RUN TestPrivateCalico I0721 22:14:24.554330 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:24.554392 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:24.554464 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:24.554533 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:24.554624 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:24.554655 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:24.554685 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:24.554723 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:24.554752 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:24.585321 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:24.588983 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.589089 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.589154 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.627815 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:24.974723 59970 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0721 22:14:24.975191 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:24.975411 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:24.975653 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:24.975720 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:24.975784 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:24.975760 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:24.977996 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:24.982193 59970 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0721 22:14:24.987586 59970 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0721 22:14:24.988130 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.988330 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.988576 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.988688 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.989474 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.990964 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:24.991083 59970 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0721 22:14:24.991607 59970 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0721 22:14:24.991829 59970 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0721 22:14:25.073195 59970 target.go:112] Terraform output is in /tmp/test3446927772/out I0721 22:14:25.079010 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:25.079076 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:25.079131 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:25.079184 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:25.079230 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:25.079259 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:25.079288 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:25.079324 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:25.079352 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:25.110894 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:25.114481 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.114626 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.114777 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.154992 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:25.502413 59970 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0721 22:14:25.503095 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:25.503129 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:25.503196 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:25.503222 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:25.503287 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:25.503333 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:25.505495 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:25.505799 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:25.505939 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:25.506107 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:25.509918 59970 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0721 22:14:25.515546 59970 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0721 22:14:25.516200 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.516411 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.516675 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.516840 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.517524 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.518606 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.518745 59970 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0721 22:14:25.519118 59970 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0721 22:14:25.519270 59970 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0721 22:14:25.523670 59970 target.go:168] Cloudformation output is in /tmp/test4160284884/out --- PASS: TestPrivateCalico (0.99s) === RUN TestPrivateCilium I0721 22:14:25.540520 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:25.540579 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:25.540636 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:25.540699 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:25.540812 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:25.540845 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:25.540890 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:25.540930 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:25.540966 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:25.569497 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:25.572682 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.572774 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.572844 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.612462 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:25.770816 59970 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0721 22:14:25.771585 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:25.772053 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:25.772092 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:25.772132 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:25.772085 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:25.772185 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:25.772326 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:25.774801 59970 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0721 22:14:25.784198 59970 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0721 22:14:25.784852 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.785072 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.785353 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.785722 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.786352 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.787979 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.788099 59970 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0721 22:14:25.788554 59970 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0721 22:14:25.788712 59970 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0721 22:14:25.868045 59970 target.go:112] Terraform output is in /tmp/test894349761/out I0721 22:14:25.872655 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:25.872723 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:25.872773 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:25.872830 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:25.872873 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:25.872901 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:25.872930 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:25.872966 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:25.872990 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:25.900337 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:25.903391 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.903485 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.903543 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:25.944486 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:26.102012 59970 executor.go:111] Tasks: 0 done / 118 total; 48 can run W0721 22:14:26.102609 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:26.102819 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:26.102825 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:26.102907 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:26.102961 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:26.102988 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:26.104706 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" },{ Key: "Name", Value: "privatecilium.example.com" }] }] } I0721 22:14:26.105100 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:26.105140 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:26.105270 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:26.109975 59970 executor.go:111] Tasks: 48 done / 118 total; 28 can run I0721 22:14:26.116071 59970 executor.go:111] Tasks: 76 done / 118 total; 36 can run I0721 22:14:26.116387 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.116566 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.116891 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.117034 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.117562 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.118957 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.119107 59970 executor.go:111] Tasks: 112 done / 118 total; 5 can run I0721 22:14:26.119549 59970 executor.go:111] Tasks: 117 done / 118 total; 1 can run I0721 22:14:26.119703 59970 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0721 22:14:26.123830 59970 target.go:168] Cloudformation output is in /tmp/test4224976295/out --- PASS: TestPrivateCilium (0.60s) === RUN TestPrivateCilium2 I0721 22:14:26.137890 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:26.137979 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:26.138091 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:26.138199 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:26.138338 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:26.138410 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:26.138477 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:26.138544 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:26.138613 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:26.170174 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0721 22:14:26.173464 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.173559 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:26.173620 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.655461 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:27.807064 59970 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0721 22:14:27.808389 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:27.808619 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:27.808912 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:27.808955 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:27.809139 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:27.809336 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:27.809487 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:27.814370 59970 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0721 22:14:27.819914 59970 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0721 22:14:27.820281 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.820514 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.820720 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.823017 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.823149 59970 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0721 22:14:27.823477 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.825065 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.825186 59970 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0721 22:14:27.825539 59970 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0721 22:14:27.906273 59970 target.go:112] Terraform output is in /tmp/test1648904072/out I0721 22:14:27.911280 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:27.911341 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:27.911390 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:27.911447 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:27.911491 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:27.911520 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:27.911549 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:27.911585 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:27.911625 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:27.940997 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.18.20 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.18.20 ********************************************************************************* I0721 22:14:27.944339 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.944433 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.944540 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:27.985840 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:28.130691 59970 executor.go:111] Tasks: 0 done / 121 total; 49 can run W0721 22:14:28.131284 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:28.131513 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:28.131666 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:28.131745 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:28.131790 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:28.131874 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:28.131921 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:28.132010 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:28.132031 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:28.132123 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecilium.example.com" },{ Key: "KubernetesCluster", Value: "privatecilium.example.com" },{ Key: "kubernetes.io/cluster/privatecilium.example.com", Value: "owned" }] }] } I0721 22:14:28.134243 59970 executor.go:111] Tasks: 49 done / 121 total; 29 can run I0721 22:14:28.143425 59970 executor.go:111] Tasks: 78 done / 121 total; 35 can run I0721 22:14:28.143867 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.144193 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.144358 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.146244 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.146394 59970 executor.go:111] Tasks: 113 done / 121 total; 6 can run I0721 22:14:28.146758 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.148114 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.148259 59970 executor.go:111] Tasks: 119 done / 121 total; 2 can run I0721 22:14:28.148668 59970 executor.go:111] Tasks: 121 done / 121 total; 0 can run I0721 22:14:28.153391 59970 target.go:168] Cloudformation output is in /tmp/test1205125513/out --- PASS: TestPrivateCilium2 (2.03s) === RUN TestPrivateCiliumAdvanced I0721 22:14:28.167081 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:28.167136 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:28.167179 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:28.167231 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:28.167315 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:28.167344 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:28.167374 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:28.167412 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:28.167440 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:28.204568 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:28.207880 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.207982 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.208045 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.247601 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:28.404399 59970 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0721 22:14:28.404860 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:28.405144 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:28.405280 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:28.405345 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:28.405441 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:28.405377 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:28.405535 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:28.408170 59970 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0721 22:14:28.418128 59970 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0721 22:14:28.418544 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.418734 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.418876 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.419113 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.420067 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.421862 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.421981 59970 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0721 22:14:28.422457 59970 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0721 22:14:28.422624 59970 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0721 22:14:28.505670 59970 target.go:112] Terraform output is in /tmp/test1988888963/out I0721 22:14:28.510552 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:28.510617 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:28.510672 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:28.510727 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:28.510771 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:28.510800 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:28.510832 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:28.510887 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:28.510918 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:28.543227 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:28.547832 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.548181 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.548404 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.588342 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:28.746839 59970 executor.go:111] Tasks: 0 done / 124 total; 54 can run W0721 22:14:28.747557 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:28.747668 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:28.747688 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:28.747900 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:28.747908 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:28.747932 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:28.748034 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:28.748082 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:28.748109 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:28.748230 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privateciliumadvanced.example.com" },{ Key: "KubernetesCluster", Value: "privateciliumadvanced.example.com" },{ Key: "kubernetes.io/cluster/privateciliumadvanced.example.com", Value: "owned" }] }] } I0721 22:14:28.750162 59970 executor.go:111] Tasks: 54 done / 124 total; 28 can run I0721 22:14:28.756390 59970 executor.go:111] Tasks: 82 done / 124 total; 36 can run I0721 22:14:28.756887 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.757089 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.757289 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.758243 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.760082 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.763322 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.763480 59970 executor.go:111] Tasks: 118 done / 124 total; 5 can run I0721 22:14:28.763917 59970 executor.go:111] Tasks: 123 done / 124 total; 1 can run I0721 22:14:28.764074 59970 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0721 22:14:28.768265 59970 target.go:168] Cloudformation output is in /tmp/test3615140697/out --- PASS: TestPrivateCiliumAdvanced (0.62s) === RUN TestPrivateCanal I0721 22:14:28.782896 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:28.782948 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:28.782995 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:28.783043 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:28.783122 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:28.783150 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:28.783178 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:28.783210 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:28.783236 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:28.809907 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0721 22:14:28.812431 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.812507 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:28.812561 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:29.969658 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:30.373368 59970 executor.go:111] Tasks: 0 done / 119 total; 49 can run W0721 22:14:30.374087 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.374077 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:30.374181 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:30.374327 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:30.375562 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.374213 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:30.375908 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:30.381144 59970 executor.go:111] Tasks: 49 done / 119 total; 28 can run I0721 22:14:30.386971 59970 executor.go:111] Tasks: 77 done / 119 total; 36 can run I0721 22:14:30.387210 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:30.387443 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:30.387581 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:30.387869 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:30.388563 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:30.391915 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:30.392181 59970 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0721 22:14:30.394980 59970 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0721 22:14:30.395183 59970 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0721 22:14:30.471604 59970 target.go:112] Terraform output is in /tmp/test1384510859/out --- PASS: TestPrivateCanal (1.69s) === RUN TestPrivateKopeio I0721 22:14:30.477268 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:30.477334 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:30.477386 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:30.477436 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:30.477530 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:30.477559 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:30.477589 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:30.477626 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:30.477654 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:30.506807 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:30.509914 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.510010 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.510074 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.550005 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:30.667303 59970 executor.go:111] Tasks: 0 done / 124 total; 48 can run W0721 22:14:30.667950 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:30.667952 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.668523 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:30.668572 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:30.668933 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:30.670640 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.671787 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:30.675363 59970 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0721 22:14:30.680372 59970 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0721 22:14:30.680795 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.680979 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.681267 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.681582 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.684369 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.687168 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.687307 59970 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0721 22:14:30.687879 59970 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0721 22:14:30.768091 59970 target.go:112] Terraform output is in /tmp/test2288779406/out --- PASS: TestPrivateKopeio (0.30s) === RUN TestUnmanaged I0721 22:14:30.772944 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:30.773009 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:30.773060 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:30.773116 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:30.773163 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:30.773191 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:30.773220 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:30.773262 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:30.773289 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:30.803452 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:30.806459 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.806547 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.806603 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.846507 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:30.951660 59970 executor.go:111] Tasks: 0 done / 106 total; 46 can run I0721 22:14:30.952284 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:30.952397 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.952515 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:30.952536 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.952543 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:30.952628 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:30.955287 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:30.959450 59970 executor.go:111] Tasks: 46 done / 106 total; 25 can run I0721 22:14:30.964350 59970 executor.go:111] Tasks: 71 done / 106 total; 31 can run I0721 22:14:30.964704 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.964897 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.964984 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.965156 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.966143 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.967543 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:30.967695 59970 executor.go:111] Tasks: 102 done / 106 total; 4 can run I0721 22:14:30.968169 59970 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0721 22:14:31.042027 59970 target.go:112] Terraform output is in /tmp/test3974718972/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I0721 22:14:31.046645 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:31.046707 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:31.046759 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:31.046814 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:31.046856 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:31.046883 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:31.046913 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:31.046952 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:31.046982 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:31.075906 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:31.078724 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.078815 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.078870 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.118449 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:31.230568 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run W0721 22:14:31.231017 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:31.231017 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:31.231369 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:31.231346 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:31.231405 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:31.231549 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:31.231603 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:31.235691 59970 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0721 22:14:31.236419 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:31.245828 59970 executor.go:111] Tasks: 70 done / 105 total; 31 can run I0721 22:14:31.246170 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.246301 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.246405 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.246539 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.247033 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.247995 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.248147 59970 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0721 22:14:31.248614 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:31.319122 59970 target.go:112] Terraform output is in /tmp/test1004006805/out --- PASS: TestPrivateSharedSubnet (0.28s) === RUN TestPrivateSharedIP I0721 22:14:31.324720 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:31.324785 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:31.324852 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:31.324918 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:31.325026 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:31.325069 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:31.325104 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:31.325148 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:31.325181 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:31.358075 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:31.361368 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.361464 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.361528 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.400876 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:31.509734 59970 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0721 22:14:31.510213 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:31.510414 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:31.510634 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:31.510773 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:31.510806 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:31.510810 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:31.510940 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:31.514394 59970 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0721 22:14:31.515542 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:31.521235 59970 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0721 22:14:31.521893 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.522118 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.522375 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.522546 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.523058 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.524652 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.524771 59970 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0721 22:14:31.525240 59970 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0721 22:14:31.525430 59970 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0721 22:14:31.603376 59970 target.go:112] Terraform output is in /tmp/test3786728077/out I0721 22:14:31.607868 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:31.607937 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:31.607991 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:31.608045 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:31.608088 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:31.608117 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:31.608146 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:31.608180 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:31.608208 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:31.633580 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:31.635857 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.635933 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.635980 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.675535 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:31.783639 59970 executor.go:111] Tasks: 0 done / 114 total; 46 can run W0721 22:14:31.784049 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:31.784139 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:31.784303 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:31.784375 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:31.784466 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:31.784487 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:31.784568 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:31.784597 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:31.784680 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:31.784590 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:31.786523 59970 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0721 22:14:31.787190 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:31.791341 59970 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0721 22:14:31.791885 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.792127 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.792383 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.792582 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.793470 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.794275 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:31.794427 59970 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0721 22:14:31.794890 59970 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0721 22:14:31.795049 59970 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0721 22:14:31.798816 59970 target.go:168] Cloudformation output is in /tmp/test1528695019/out --- PASS: TestPrivateSharedIP (0.49s) === RUN TestPrivateDns1 I0721 22:14:31.816936 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:31.816992 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:31.817038 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:31.817089 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:31.817173 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:31.817205 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:31.817235 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:31.817271 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:31.817299 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:31.848190 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:31.848284 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:31.848354 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.017400 59970 executor.go:111] Tasks: 0 done / 118 total; 47 can run W0721 22:14:32.017955 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:32.017957 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:32.018163 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:32.018149 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:32.020671 59970 executor.go:111] Tasks: 47 done / 118 total; 24 can run I0721 22:14:32.020966 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0721 22:14:32.020991 59970 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0721 22:14:32.021044 59970 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0721 22:14:32.021052 59970 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0721 22:14:32.021315 59970 executor.go:111] Tasks: 71 done / 118 total; 35 can run I0721 22:14:32.021760 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.022008 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.023420 59970 executor.go:111] Tasks: 106 done / 118 total; 5 can run W0721 22:14:32.023480 59970 api.go:53] PageSize not implemented W0721 22:14:32.023569 59970 api.go:53] PageSize not implemented W0721 22:14:32.023501 59970 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0721 22:14:32.023601 59970 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns1.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns1-example-c-lq96ht","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns1.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"utility-us-test-1a.privatedns1.example.com","Owner":"John Doe","SubnetType":"Utility","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns1.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns1.example.com","Lifecycle":"Sync","ID":null,"CIDR":"172.20.0.0/16","AmazonIPv6":true,"IPv6CIDR":null,"EnableDNSHostnames":true,"EnableDNSSupport":true,"Shared":false,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api-elb.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns1.example.com","Name":"api.privatedns1.example.com","Owner":"John Doe","foo/bar":"fib+baz","kubernetes.io/cluster/privatedns1.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0721 22:14:32.026498 59970 executor.go:111] Tasks: 111 done / 118 total; 5 can run I0721 22:14:32.026662 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.026826 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.027920 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.027982 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.028050 59970 executor.go:111] Tasks: 116 done / 118 total; 2 can run I0721 22:14:32.028365 59970 executor.go:111] Tasks: 118 done / 118 total; 0 can run I0721 22:14:32.112165 59970 target.go:112] Terraform output is in /tmp/test4061812082/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I0721 22:14:32.120209 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:32.120283 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:32.120344 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:32.120402 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:32.120450 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:32.120480 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:32.120512 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:32.120546 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:32.120573 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:32.152676 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.152772 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.152832 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.293193 59970 executor.go:111] Tasks: 0 done / 114 total; 45 can run W0721 22:14:32.293647 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:32.293689 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:32.294114 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:32.294186 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:32.301542 59970 executor.go:111] Tasks: 45 done / 114 total; 22 can run I0721 22:14:32.301855 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0721 22:14:32.301896 59970 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0721 22:14:32.301983 59970 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0721 22:14:32.302002 59970 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0721 22:14:32.302157 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:32.302367 59970 executor.go:111] Tasks: 67 done / 114 total; 35 can run I0721 22:14:32.302880 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.303168 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.305650 59970 executor.go:111] Tasks: 102 done / 114 total; 5 can run W0721 22:14:32.305785 59970 api.go:53] PageSize not implemented W0721 22:14:32.305856 59970 api.go:53] PageSize not implemented W0721 22:14:32.305826 59970 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} W0721 22:14:32.305889 59970 bootstrapscript.go:91] Task did not have an address: *awstasks.ClassicLoadBalancer {"Name":"api.privatedns2.example.com","Lifecycle":"Sync","LoadBalancerName":"api-privatedns2-example-c-6jft30","DNSName":null,"HostedZoneId":null,"Subnets":[{"Name":"utility-us-test-1a.privatedns2.example.com","ShortName":"utility-us-test-1a","Lifecycle":"Sync","ID":null,"VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"AmazonIPv6CIDR":null,"AvailabilityZone":"us-test-1a","CIDR":"172.20.4.0/22","IPv6CIDR":null,"Shared":false,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"utility-us-test-1a.privatedns2.example.com","SubnetType":"Utility","kubernetes.io/cluster/privatedns2.example.com":"owned","kubernetes.io/role/elb":"1"}}],"SecurityGroups":[{"Name":"api-elb.privatedns2.example.com","Lifecycle":"Sync","ID":null,"Description":"Security group for api ELB","VPC":{"Name":"privatedns2.example.com","Lifecycle":"Sync","ID":"vpc-12345678","CIDR":"172.20.0.0/16","AmazonIPv6":null,"IPv6CIDR":null,"EnableDNSHostnames":null,"EnableDNSSupport":true,"Shared":true,"Tags":null,"AssociateExtraCIDRBlocks":null},"RemoveExtraRules":["port=443"],"Shared":null,"Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api-elb.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"}}],"Listeners":{"443":{"InstancePort":443,"SSLCertificateID":""}},"Scheme":null,"HealthCheck":{"Target":"SSL:443","HealthyThreshold":2,"UnhealthyThreshold":2,"Interval":10,"Timeout":5},"AccessLog":{"EmitInterval":null,"Enabled":false,"S3BucketName":null,"S3BucketPrefix":null},"ConnectionDraining":{"Enabled":true,"Timeout":300},"ConnectionSettings":{"IdleTimeout":300},"CrossZoneLoadBalancing":{"Enabled":false},"SSLCertificateID":"","Tags":{"KubernetesCluster":"privatedns2.example.com","Name":"api.privatedns2.example.com","kubernetes.io/cluster/privatedns2.example.com":"owned"},"ForAPIServer":true,"Shared":null} I0721 22:14:32.311085 59970 executor.go:111] Tasks: 107 done / 114 total; 5 can run I0721 22:14:32.311371 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.311649 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.313158 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.313979 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.314105 59970 executor.go:111] Tasks: 112 done / 114 total; 2 can run I0721 22:14:32.314433 59970 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0721 22:14:32.390304 59970 target.go:112] Terraform output is in /tmp/test3384792428/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I0721 22:14:32.394877 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:32.394943 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:32.394996 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:32.395053 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:32.395098 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:32.395128 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:32.395158 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:32.395193 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:32.395221 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:32.420311 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0721 22:14:32.423478 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:32.423588 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.553034 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:33.662594 59970 executor.go:111] Tasks: 0 done / 91 total; 47 can run W0721 22:14:33.663704 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:33.663866 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:33.663883 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:33.663951 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:33.667949 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:33.668001 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:33.670633 59970 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0721 22:14:33.676356 59970 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0721 22:14:33.677978 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.678304 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.682092 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.683267 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.683399 59970 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0721 22:14:33.683785 59970 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0721 22:14:33.735866 59970 target.go:112] Terraform output is in /tmp/test4268552842/out --- PASS: TestDiscoveryFeatureGate (1.35s) === RUN TestVFSServiceAccountIssuerDiscovery I0721 22:14:33.740574 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:33.740638 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:33.740689 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:33.740748 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:33.740790 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:33.740817 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:33.740845 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:33.740880 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:33.740907 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:33.765346 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:33.768275 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.768368 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.807577 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:33.911864 59970 executor.go:111] Tasks: 0 done / 89 total; 46 can run I0721 22:14:33.912200 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:33.912234 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:33.912337 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:33.912473 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:33.912875 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:33.912893 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:33.920387 59970 executor.go:111] Tasks: 46 done / 89 total; 20 can run I0721 22:14:33.925979 59970 executor.go:111] Tasks: 66 done / 89 total; 21 can run I0721 22:14:33.926426 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.926651 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.927753 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.928838 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:33.928962 59970 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0721 22:14:33.929233 59970 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0721 22:14:33.986174 59970 target.go:112] Terraform output is in /tmp/test136460151/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.25s) === RUN TestAWSLBController I0721 22:14:33.990846 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:33.990913 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:33.990967 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:33.991022 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:33.991135 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:33.991168 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:33.991206 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:33.991247 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:33.991288 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:34.011797 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:34.013447 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:34.013505 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:34.051912 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:35.377204 59970 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0721 22:14:35.377820 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:35.377866 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:35.383763 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:35.383794 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:35.383816 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:35.383893 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:35.397634 59970 executor.go:111] Tasks: 50 done / 95 total; 22 can run I0721 22:14:35.403007 59970 executor.go:111] Tasks: 72 done / 95 total; 21 can run I0721 22:14:35.403428 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:35.403634 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:35.404621 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:35.406023 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:35.406146 59970 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0721 22:14:35.406459 59970 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0721 22:14:35.463646 59970 target.go:112] Terraform output is in /tmp/test2109815087/out --- PASS: TestAWSLBController (1.50s) === RUN TestManyAddons I0721 22:14:35.485995 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:35.486062 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:35.486116 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:35.486168 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:35.486273 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:35.486303 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:35.486333 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:35.486373 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:35.486401 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:35.512356 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:35.514941 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:35.515015 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:35.555740 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:37.067294 59970 executor.go:111] Tasks: 0 done / 94 total; 51 can run I0721 22:14:37.068523 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:37.068550 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:37.069417 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:37.069454 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:37.071024 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:37.073439 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:37.089507 59970 executor.go:111] Tasks: 51 done / 94 total; 19 can run I0721 22:14:37.095405 59970 executor.go:111] Tasks: 70 done / 94 total; 22 can run I0721 22:14:37.095662 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:37.096003 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:37.096983 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:37.098825 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:37.098954 59970 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0721 22:14:37.099284 59970 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0721 22:14:37.156590 59970 target.go:112] Terraform output is in /tmp/test555297383/out --- PASS: TestManyAddons (1.69s) === RUN TestManyAddonsCCMIRSA I0721 22:14:37.180612 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:37.180686 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:37.180739 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:37.180794 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:37.180884 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:37.180913 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:37.180944 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:37.180979 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:37.181005 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:37.208573 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:37.212192 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:37.212292 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:37.251911 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:38.850555 59970 executor.go:111] Tasks: 0 done / 110 total; 60 can run I0721 22:14:38.850873 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:38.850907 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:38.850987 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:38.851260 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:38.853308 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:38.854933 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:38.871175 59970 executor.go:111] Tasks: 60 done / 110 total; 26 can run I0721 22:14:38.877136 59970 executor.go:111] Tasks: 86 done / 110 total; 22 can run I0721 22:14:38.877439 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:38.877723 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:38.879066 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:38.880119 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:38.880253 59970 executor.go:111] Tasks: 108 done / 110 total; 2 can run I0721 22:14:38.880613 59970 executor.go:111] Tasks: 110 done / 110 total; 0 can run I0721 22:14:38.949791 59970 target.go:112] Terraform output is in /tmp/test1710928886/out --- PASS: TestManyAddonsCCMIRSA (1.80s) === RUN TestCCM I0721 22:14:38.979270 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:38.979345 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:38.979396 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:38.979450 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:38.979550 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:38.979581 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:38.979612 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:38.979648 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:38.979677 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:39.005759 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:39.008725 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:39.008823 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:39.048387 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:40.575078 59970 executor.go:111] Tasks: 0 done / 95 total; 52 can run I0721 22:14:40.575822 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:40.575865 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:40.575962 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:40.575933 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:40.576065 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:40.576140 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:40.592454 59970 executor.go:111] Tasks: 52 done / 95 total; 19 can run I0721 22:14:40.601902 59970 executor.go:111] Tasks: 71 done / 95 total; 22 can run I0721 22:14:40.602502 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.602744 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.603920 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.605203 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.605334 59970 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0721 22:14:40.605696 59970 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0721 22:14:40.659830 59970 target.go:112] Terraform output is in /tmp/test2450282934/out --- PASS: TestCCM (1.71s) === RUN TestExternalDNS I0721 22:14:40.684490 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:40.684555 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:40.684604 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:40.684661 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:40.684745 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:40.684774 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:40.684806 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:40.684843 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:40.684872 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:40.708990 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:40.711822 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.711920 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.754081 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:40.858425 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:40.858754 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:40.858788 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:40.858916 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:40.858871 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:40.859161 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:40.859356 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:40.861713 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:40.871455 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:40.872052 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.872346 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.873461 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.875303 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.875429 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:40.875885 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:40.930622 59970 target.go:112] Terraform output is in /tmp/test2042518602/out I0721 22:14:40.935160 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:40.935228 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:40.935280 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:40.935336 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:40.935380 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:40.935411 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:40.935440 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:40.935477 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:40.935505 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:40.959396 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:40.962242 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:40.962339 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.001562 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:41.109767 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0721 22:14:41.110245 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:41.110361 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:41.110384 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:41.110502 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:41.110586 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:41.113178 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:41.113452 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:41.113481 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:41.113673 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0721 22:14:41.117672 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:41.122894 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:41.123123 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.123480 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.124155 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.125924 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.126056 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:41.126309 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:41.129789 59970 target.go:168] Cloudformation output is in /tmp/test729419147/out --- PASS: TestExternalDNS (0.46s) === RUN TestExternalDNSIRSA I0721 22:14:41.139858 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:41.139912 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:41.139956 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:41.140004 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:41.140048 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:41.140076 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:41.140106 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:41.140141 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:41.140170 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:41.169446 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:41.172340 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.172446 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.211046 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:41.334094 59970 executor.go:111] Tasks: 0 done / 91 total; 47 can run W0721 22:14:41.334660 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:41.334800 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:41.334779 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:41.334834 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:41.334925 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:41.335026 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:41.337706 59970 executor.go:111] Tasks: 47 done / 91 total; 21 can run I0721 22:14:41.342682 59970 executor.go:111] Tasks: 68 done / 91 total; 21 can run I0721 22:14:41.343193 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.343425 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.344627 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.346755 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.347013 59970 executor.go:111] Tasks: 89 done / 91 total; 2 can run I0721 22:14:41.347685 59970 executor.go:111] Tasks: 91 done / 91 total; 0 can run I0721 22:14:41.403914 59970 target.go:112] Terraform output is in /tmp/test1069216161/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestSharedSubnet I0721 22:14:41.411746 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:41.411802 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:41.411853 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:41.411922 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:41.411992 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:41.412041 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:41.412096 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:41.412152 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:41.412195 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:41.437124 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:41.440022 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.440111 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.478859 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:41.585652 59970 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0721 22:14:41.586300 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:41.586297 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:41.586338 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:41.586429 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:41.586810 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:41.586876 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:41.588907 59970 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0721 22:14:41.589258 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:41.598252 59970 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0721 22:14:41.598839 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.599103 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.599930 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.601604 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.601759 59970 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0721 22:14:41.602063 59970 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0721 22:14:41.652351 59970 target.go:112] Terraform output is in /tmp/test3426701399/out --- PASS: TestSharedSubnet (0.25s) === RUN TestSharedVPC I0721 22:14:41.656975 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:41.657028 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:41.657076 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:41.657126 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:41.657165 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:41.657194 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:41.657223 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:41.657256 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:41.657281 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:41.681880 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:41.684723 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.684815 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.724190 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:41.828360 59970 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0721 22:14:41.828603 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:41.828630 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:41.828711 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:41.828757 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:41.829135 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:41.829346 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:41.831788 59970 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0721 22:14:41.832123 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:41.841169 59970 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0721 22:14:41.841449 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.841614 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.842740 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.843775 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:41.843898 59970 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0721 22:14:41.844243 59970 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0721 22:14:41.895986 59970 target.go:112] Terraform output is in /tmp/test2241986017/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I0721 22:14:41.901479 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:41.901539 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:41.901588 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:41.901643 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:41.901686 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:41.901714 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:41.901744 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:41.901780 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:41.901810 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:41.934917 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:41.938112 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:41.938203 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:41.938259 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:41.938316 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:41.978095 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:42.066724 59970 executor.go:111] Tasks: 0 done / 94 total; 48 can run W0721 22:14:42.067263 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:42.067393 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:42.067569 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:42.067658 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:42.067704 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:42.067743 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0721 22:14:42.067868 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:42.067883 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:42.070104 59970 executor.go:111] Tasks: 48 done / 94 total; 15 can run I0721 22:14:42.078983 59970 executor.go:111] Tasks: 63 done / 94 total; 27 can run I0721 22:14:42.079225 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.079385 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.079539 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.079733 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.080675 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.081997 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.082307 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.082418 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:42.082539 59970 executor.go:111] Tasks: 90 done / 94 total; 4 can run I0721 22:14:42.083118 59970 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0721 22:14:42.162944 59970 target.go:112] Terraform output is in /tmp/test3828036936/out --- PASS: TestExistingIAM (0.27s) === RUN TestPhaseNetwork I0721 22:14:42.167488 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:42.167551 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:42.167602 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:42.167657 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:42.167700 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:42.167728 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:42.167758 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:42.167813 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:42.167843 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:42.195983 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:42.198871 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.199032 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.199092 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.239982 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:42.346551 59970 executor.go:111] Tasks: 0 done / 117 total; 47 can run I0721 22:14:42.346890 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:42.347014 59970 executor.go:111] Tasks: 47 done / 117 total; 28 can run I0721 22:14:42.347500 59970 executor.go:111] Tasks: 75 done / 117 total; 36 can run I0721 22:14:42.347946 59970 executor.go:111] Tasks: 111 done / 117 total; 5 can run I0721 22:14:42.348200 59970 executor.go:111] Tasks: 116 done / 117 total; 1 can run I0721 22:14:42.348378 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0721 22:14:42.362504 59970 target.go:112] Terraform output is in /tmp/test575211313/out --- PASS: TestPhaseNetwork (0.20s) === RUN TestExternalLoadBalancer I0721 22:14:42.363561 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:42.363619 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:42.363749 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:42.363805 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:42.363850 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:42.363878 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:42.363912 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:42.363947 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:42.363972 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:42.389592 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:42.392460 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.392576 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.433706 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:42.542922 59970 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0721 22:14:42.543282 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:42.543622 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:42.543720 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:42.543753 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:42.543843 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:42.544141 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:42.550940 59970 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0721 22:14:42.556391 59970 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0721 22:14:42.556613 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.556855 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.557483 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.557959 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.558022 59970 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0721 22:14:42.558213 59970 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0721 22:14:42.613343 59970 target.go:112] Terraform output is in /tmp/test3637823503/out I0721 22:14:42.617554 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:42.617614 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:42.617663 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:42.617713 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:42.617754 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:42.617803 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:42.617840 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:42.617876 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:42.617903 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:42.643036 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:42.645873 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.645981 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.684644 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:42.790782 59970 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0721 22:14:42.791376 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:42.791638 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:42.791673 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:42.791756 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:42.791769 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:42.793313 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:42.793349 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:42.793521 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W0721 22:14:42.794395 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:42.798821 59970 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0721 22:14:42.804643 59970 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0721 22:14:42.804881 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.805352 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.806403 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.807902 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.808040 59970 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0721 22:14:42.808376 59970 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0721 22:14:42.811534 59970 target.go:168] Cloudformation output is in /tmp/test2301699321/out --- PASS: TestExternalLoadBalancer (0.46s) === RUN TestPhaseIAM integration_test.go:633: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:641: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0721 22:14:42.823041 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:42.823123 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:42.823248 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:42.823344 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:42.823451 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:42.823486 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:42.824187 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:42.824237 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:42.824281 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:42.858424 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:42.861614 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.861713 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.861774 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.861831 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.861882 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.861936 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.861994 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:42.901951 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:43.011804 59970 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0721 22:14:43.012306 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:43.012400 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:43.012403 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:43.012473 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:43.012477 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:43.012450 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:43.012922 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:43.012989 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:43.015697 59970 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0721 22:14:43.025024 59970 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0721 22:14:43.025342 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.025615 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.025868 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.026118 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.027219 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.027832 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.028212 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.028282 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.028399 59970 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0721 22:14:43.028766 59970 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0721 22:14:43.111952 59970 target.go:112] Terraform output is in /tmp/test325738924/out I0721 22:14:43.116865 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:43.116931 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:43.116993 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:43.117048 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:43.117091 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:43.117120 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:43.117149 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:43.117183 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:43.117212 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:43.149851 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:43.152792 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.152881 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.152941 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.152995 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.153045 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.153090 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.153133 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.191739 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:43.301729 59970 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0721 22:14:43.302047 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:43.302058 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:43.302201 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:43.302656 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:43.302764 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0721 22:14:43.302819 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:43.302851 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:43.302798 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:43.302943 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:43.302879 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:43.302996 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0721 22:14:43.304655 59970 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0721 22:14:43.313625 59970 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0721 22:14:43.314204 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.314397 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.314715 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.314979 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.315831 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.317327 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.318180 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.318345 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.318487 59970 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0721 22:14:43.318873 59970 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0721 22:14:43.323481 59970 target.go:168] Cloudformation output is in /tmp/test1301473050/out --- PASS: TestMixedInstancesASG (0.52s) === RUN TestMixedInstancesSpotASG I0721 22:14:43.341607 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:43.341658 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:43.341702 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:43.341753 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:43.341830 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:43.341858 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:43.341903 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:43.341937 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:43.341963 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:43.378483 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:43.381599 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.381701 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.381764 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.381823 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.381877 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.381927 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.381975 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.420888 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:43.529624 59970 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0721 22:14:43.529957 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:43.530390 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:43.530458 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:43.530618 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:43.530603 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:43.530784 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:43.530830 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:43.530956 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:43.532503 59970 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0721 22:14:43.543022 59970 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0721 22:14:43.543659 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.543978 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.544204 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.544491 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.545675 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.546146 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.546334 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.546666 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.546817 59970 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0721 22:14:43.547448 59970 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0721 22:14:43.628130 59970 target.go:112] Terraform output is in /tmp/test3796962094/out I0721 22:14:43.633420 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:43.633488 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:43.633540 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:43.633595 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:43.633642 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:43.633673 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:43.633705 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:43.633740 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:43.633769 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:43.671215 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:43.674391 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.674533 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.674604 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.674664 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.674719 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.675132 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.675201 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.714546 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:43.822503 59970 executor.go:111] Tasks: 0 done / 104 total; 50 can run W0721 22:14:43.823013 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:43.823010 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:43.823243 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:43.823284 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:43.823242 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:43.823403 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:43.823425 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:43.823540 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "mixedinstances.example.com" },{ Key: "KubernetesCluster", Value: "mixedinstances.example.com" },{ Key: "kubernetes.io/cluster/mixedinstances.example.com", Value: "owned" }] }] } I0721 22:14:43.823831 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:43.823877 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:43.823998 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:43.826220 59970 executor.go:111] Tasks: 50 done / 104 total; 23 can run I0721 22:14:43.835846 59970 executor.go:111] Tasks: 73 done / 104 total; 27 can run I0721 22:14:43.836295 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.836466 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.836672 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.837027 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.838218 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.839012 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.839127 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.839537 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:43.839678 59970 executor.go:111] Tasks: 100 done / 104 total; 4 can run I0721 22:14:43.840104 59970 executor.go:111] Tasks: 104 done / 104 total; 0 can run I0721 22:14:43.845120 59970 target.go:168] Cloudformation output is in /tmp/test1484259600/out --- PASS: TestMixedInstancesSpotASG (0.52s) === RUN TestContainerd I0721 22:14:43.865892 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:43.865950 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:43.866005 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:43.866060 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:43.866150 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:43.866180 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:43.866210 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:43.866247 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:43.866274 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:43.889999 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:43.892865 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:43.892963 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:43.932411 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:44.040621 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run W0721 22:14:44.041151 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:44.041314 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:44.041309 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:44.041510 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:44.041374 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0721 22:14:44.041845 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:44.041844 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:44.041929 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:44.042173 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } I0721 22:14:44.046763 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:44.052898 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:44.053411 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.053666 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.054656 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.056251 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.056386 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:44.056624 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:44.059618 59970 target.go:168] Cloudformation output is in /tmp/test1035270966/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I0721 22:14:44.070521 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:44.070612 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:44.070729 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:44.070814 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:44.070961 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:44.071137 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:44.071292 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:44.071442 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:44.071549 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:44.098939 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:44.102004 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.102115 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.141202 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:44.250148 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:44.250743 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:44.250781 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:44.250924 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:44.252786 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:44.252991 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:44.253013 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:44.253137 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "containerd.example.com" },{ Key: "KubernetesCluster", Value: "containerd.example.com" },{ Key: "kubernetes.io/cluster/containerd.example.com", Value: "owned" }] }] } W0721 22:14:44.254122 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:44.254340 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:44.258617 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:44.264141 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:44.264436 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.264859 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.266061 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.266858 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.266991 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:44.267307 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:44.270598 59970 target.go:168] Cloudformation output is in /tmp/test4164687618/out --- PASS: TestContainerdCustom (0.21s) === RUN TestDockerCustom I0721 22:14:44.281037 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:44.281185 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:44.281347 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:44.281667 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:44.281865 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:44.281933 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:44.282003 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:44.282069 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:44.282115 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:44.310694 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:44.313803 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.313905 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.353221 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:44.467098 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:44.467583 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" W0721 22:14:44.467661 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:44.467618 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:44.467733 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:44.467889 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:44.467990 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:44.468090 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:44.468122 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:44.468267 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "docker.example.com" },{ Key: "KubernetesCluster", Value: "docker.example.com" },{ Key: "kubernetes.io/cluster/docker.example.com", Value: "owned" }] }] } I0721 22:14:44.470615 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:44.475783 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:44.475997 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.476167 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.479354 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.482887 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:44.483046 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:44.483343 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:44.486239 59970 target.go:168] Cloudformation output is in /tmp/test798142560/out --- PASS: TestDockerCustom (0.22s) === RUN TestAPIServerNodes I0721 22:14:44.496876 59970 featureflag.go:162] FeatureFlag "APIServerNodes"=true I0721 22:14:44.497111 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:44.497164 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:44.497252 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:44.497304 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:44.497347 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:44.497375 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:44.497407 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:44.497438 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:44.497468 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:44.522248 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0721 22:14:44.524611 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.524680 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.524727 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.567501 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:44.732585 59970 executor.go:111] Tasks: 0 done / 98 total; 48 can run W0721 22:14:44.733155 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:44.733442 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:44.733510 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:44.733545 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0721 22:14:44.733678 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target W0721 22:14:44.733657 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:44.733724 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:44.734534 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } W0721 22:14:44.735012 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:44.733689 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse I0721 22:14:44.740321 59970 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0721 22:14:44.745884 59970 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0721 22:14:44.746151 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.746336 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.746540 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.747967 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.748176 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.749057 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.749195 59970 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0721 22:14:44.749519 59970 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0721 22:14:44.753407 59970 target.go:168] Cloudformation output is in /tmp/test3662052699/out I0721 22:14:44.769827 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:44.769925 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:44.770016 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:44.770115 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:44.770259 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:44.770292 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:44.770322 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:44.770357 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:44.770384 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:44.796432 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.4 ********************************************************************************* I0721 22:14:44.799461 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.799549 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.799609 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:44.839788 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:44.998706 59970 executor.go:111] Tasks: 0 done / 98 total; 48 can run W0721 22:14:44.999184 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:44.999457 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0721 22:14:44.999611 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:44.999498 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0721 22:14:44.999676 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:44.999686 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:44.999689 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:45.001978 59970 executor.go:111] Tasks: 48 done / 98 total; 24 can run I0721 22:14:45.009336 59970 executor.go:111] Tasks: 72 done / 98 total; 23 can run I0721 22:14:45.009614 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.009863 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.010120 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.010846 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.011383 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.012856 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.012974 59970 executor.go:111] Tasks: 95 done / 98 total; 3 can run I0721 22:14:45.013365 59970 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0721 22:14:45.081786 59970 target.go:112] Terraform output is in /tmp/test1145084618/out I0721 22:14:45.086553 59970 featureflag.go:162] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.59s) === RUN TestNTHQueueProcessor I0721 22:14:45.086906 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:45.086966 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:45.087016 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:45.087066 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:45.087160 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:45.087190 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:45.087220 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:45.087256 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:45.087283 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:45.111917 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0721 22:14:45.114848 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:45.114943 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.538756 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:46.664561 59970 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0721 22:14:46.665136 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:46.665183 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:46.665293 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:46.665475 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:46.666838 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:46.667768 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:46.672963 59970 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0721 22:14:46.678672 59970 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0721 22:14:46.679196 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.679433 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.680270 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.682057 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.682268 59970 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0721 22:14:46.682695 59970 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0721 22:14:46.682909 59970 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0721 22:14:46.748524 59970 target.go:112] Terraform output is in /tmp/test3133447033/out I0721 22:14:46.753543 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:46.753612 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:46.753665 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:46.753723 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:46.753768 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:46.753797 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:46.753827 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:46.753863 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:46.753892 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:46.778368 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0721 22:14:46.781305 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.781401 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.821663 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* Kops support for CloudFormation is deprecated and will be removed in a future release. ********************************************************************************* I0721 22:14:46.947094 59970 executor.go:111] Tasks: 0 done / 98 total; 46 can run W0721 22:14:46.947327 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:46.949144 59970 dnszone.go:316] Check for existing route53 zone to re-use with name "" I0721 22:14:46.949209 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:46.949321 59970 dnszone.go:323] Existing zone "example.com." found; will configure cloudformation to reuse W0721 22:14:46.950123 59970 warmpool.go:131] ASG warm pool is not supported by the cloudformation target I0721 22:14:46.950527 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0721 22:14:46.950686 59970 sshkey.go:235] Cloudformation does not manage SSH keys; pre-creating SSH key I0721 22:14:46.950710 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:46.950837 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:46.955922 59970 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0721 22:14:46.961441 59970 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0721 22:14:46.961959 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.962174 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.963119 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.964810 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:46.964960 59970 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0721 22:14:46.965232 59970 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0721 22:14:46.965450 59970 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0721 22:14:46.969328 59970 target.go:168] Cloudformation output is in /tmp/test3402180682/out --- PASS: TestNTHQueueProcessor (1.90s) === RUN TestCustomIRSA I0721 22:14:46.984110 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:46.984205 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:46.984299 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:46.984391 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:46.984563 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:46.984623 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:46.984680 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:46.984746 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:46.984796 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:47.013357 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0721 22:14:47.016620 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:47.016713 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:47.056851 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:48.328831 59970 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0721 22:14:48.329426 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:48.329793 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:48.329863 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:48.330077 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:48.330109 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:48.330200 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:48.345516 59970 executor.go:111] Tasks: 51 done / 99 total; 25 can run I0721 22:14:48.350976 59970 executor.go:111] Tasks: 76 done / 99 total; 21 can run I0721 22:14:48.351222 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.351540 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.354602 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.358196 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.358347 59970 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0721 22:14:48.358692 59970 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0721 22:14:48.420719 59970 target.go:112] Terraform output is in /tmp/test3789670339/out --- PASS: TestCustomIRSA (1.46s) === RUN TestCustomIRSA119 I0721 22:14:48.442702 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:48.442776 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:48.442832 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:48.442888 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:48.442933 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:48.442961 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:48.442996 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:48.443031 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:48.443061 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:48.470915 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.19.16 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.19.16 ********************************************************************************* I0721 22:14:48.473931 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.474020 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.515691 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:48.618658 59970 executor.go:111] Tasks: 0 done / 97 total; 49 can run W0721 22:14:48.619346 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:48.619571 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:48.619597 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:48.619618 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:48.619782 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0721 22:14:48.619984 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:48.622013 59970 executor.go:111] Tasks: 49 done / 97 total; 25 can run I0721 22:14:48.627283 59970 executor.go:111] Tasks: 74 done / 97 total; 21 can run I0721 22:14:48.627793 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.627988 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.629156 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.630126 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.630249 59970 executor.go:111] Tasks: 95 done / 97 total; 2 can run I0721 22:14:48.630588 59970 executor.go:111] Tasks: 97 done / 97 total; 0 can run I0721 22:14:48.690721 59970 target.go:112] Terraform output is in /tmp/test4230544780/out --- PASS: TestCustomIRSA119 (0.25s) === RUN TestClusterNameDigit I0721 22:14:48.697723 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:48.697811 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:48.697871 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:48.697924 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:48.697975 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:48.698008 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:48.698038 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:48.698079 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:48.698107 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:48.724943 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0721 22:14:48.728088 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.728183 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.768544 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:48.875192 59970 executor.go:111] Tasks: 0 done / 94 total; 48 can run W0721 22:14:48.875701 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target W0721 22:14:48.875856 59970 warmpool.go:124] ASG warm pool is not supported by the terraform target I0721 22:14:48.875855 59970 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0721 22:14:48.875904 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:48.876001 59970 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0721 22:14:48.876206 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:48.879247 59970 executor.go:111] Tasks: 48 done / 94 total; 23 can run I0721 22:14:48.887415 59970 executor.go:111] Tasks: 71 done / 94 total; 21 can run I0721 22:14:48.887907 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.888136 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.889282 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.890153 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.890275 59970 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0721 22:14:48.890589 59970 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0721 22:14:48.951648 59970 target.go:112] Terraform output is in /tmp/test2326315821/out --- PASS: TestClusterNameDigit (0.26s) === RUN TestLifecycleMinimalAWS I0721 22:14:48.956672 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:48.956730 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:48.956778 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:48.956829 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:48.956872 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:48.956900 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:48.956933 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:48.956967 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:48.956994 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:48.957061 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0721 22:14:48.980288 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:48.983042 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:48.983139 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0721 22:14:49.024465 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:49.129042 59970 iamrole.go:91] ListRoles: { } I0721 22:14:49.130675 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:49.131209 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0721 22:14:49.131312 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:49.131357 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" }] } I0721 22:14:49.131670 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0721 22:14:49.132731 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0721 22:14:49.133305 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0721 22:14:49.131527 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.134311 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0721 22:14:49.134618 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.135042 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I0721 22:14:49.135190 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.135309 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:49.135432 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:49.138060 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.138202 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.138356 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:49.138449 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:49.138514 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:49.138560 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:49.138623 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.138833 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:49.138897 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:49.138952 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:49.138996 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:49.139125 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:49.139304 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:49.139576 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0721 22:14:49.139635 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.139650 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0721 22:14:49.139818 59970 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0721 22:14:49.139764 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.139889 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:49.139985 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0721 22:14:49.140030 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0721 22:14:49.139974 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0721 22:14:49.140289 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:49.140514 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:49.140581 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0721 22:14:49.140754 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.140866 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:49.140924 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0721 22:14:49.141157 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.141402 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:49.141415 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:49.141468 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I0721 22:14:49.141564 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:49.141673 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:49.141710 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:49.141756 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:49.141841 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:49.141895 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:49.141950 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:49.142005 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:49.142054 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:49.142101 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:49.142761 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0721 22:14:49.145952 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:49.146210 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0721 22:14:49.146462 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.146696 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.146842 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.146983 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0721 22:14:49.147122 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" }] }] } I0721 22:14:49.147333 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.147463 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.147540 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:49.147665 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.147750 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.147854 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:49.147916 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:49.148006 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.148192 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0721 22:14:49.148336 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.148467 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:49.148518 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0721 22:14:49.148561 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.148667 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.148792 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.148997 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.149167 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.149296 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.149368 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0721 22:14:49.151930 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:49.152210 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:49.152274 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:49.152312 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:49.152379 59970 dns.go:238] Pre-creating DNS records I0721 22:14:49.152410 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:49.158335 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:49.161061 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.161171 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0721 22:14:49.199740 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:49.310150 59970 iamrole.go:91] ListRoles: { } I0721 22:14:49.311777 59970 executor.go:111] Tasks: 0 done / 86 total; 44 can run I0721 22:14:49.312208 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:49.312250 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.312563 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.312761 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:49.312808 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:49.312868 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:49.312907 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:49.312977 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:49.313096 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.313225 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.315770 59970 executor.go:111] Tasks: 44 done / 86 total; 19 can run I0721 22:14:49.315943 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0721 22:14:49.316364 59970 iamrole.go:163] ListAttachedRolePolicies: masters.minimal.example.com I0721 22:14:49.316260 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:49.316476 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal.example.com I0721 22:14:49.316539 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.316813 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.317145 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:49.317395 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0721 22:14:49.317813 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0721 22:14:49.318317 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:49.318660 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:49.318923 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:49.325829 59970 executor.go:111] Tasks: 63 done / 86 total; 21 can run I0721 22:14:49.326190 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.326368 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.327000 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.327153 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.327326 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.327389 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.327505 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:49.329948 59970 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0721 22:14:49.330308 59970 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0721 22:14:49.333121 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:49.333190 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:49.333227 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:49.333259 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:49.333300 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:49.333336 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:49.333369 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:49.333402 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:49.333442 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:49.333476 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:49.333513 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:49.333554 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:49.333589 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:49.333620 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:49.333653 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:49.333686 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:49.333721 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:49.333756 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:49.333793 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:49.333826 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:49.333860 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:49.333893 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:49.333948 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:49.334028 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:49.334104 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:49.335701 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:49.335735 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:49.335776 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.335830 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0721 22:14:49.335884 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.335944 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.335994 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.336051 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.336099 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.336139 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0721 22:14:49.336183 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.336234 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0721 22:14:49.336277 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0721 22:14:49.336324 59970 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0721 22:14:49.336352 59970 api.go:53] PageSize not implemented I0721 22:14:49.336393 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:49.336428 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:49.336448 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:49.336477 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:49.336498 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:49.336552 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:49.336589 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:49.336644 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:49.336683 59970 iamrole.go:91] ListRoles: { } I0721 22:14:49.336739 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:49.336787 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:49.336822 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:49.337464 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:49.337486 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0721 22:14:49.337614 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0721 22:14:49.337674 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-group:nodes.minimal.example.com ok I0721 22:14:49.337622 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0721 22:14:49.337736 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } volume:vol-1 ok I0721 22:14:49.337808 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:49.337925 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok iam-instance-profile:masters.minimal.example.com ok I0721 22:14:49.338005 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:49.337928 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok I0721 22:14:49.338118 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0721 22:14:49.338275 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:49.338302 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:49.338342 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0721 22:14:49.338401 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0721 22:14:49.338596 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:49.338614 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0721 22:14:49.338661 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0721 22:14:49.338786 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:49.338815 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0721 22:14:49.338913 59970 iamrole.go:182] ListAttachedRolePolicies: masters.minimal.example.com I0721 22:14:49.338932 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0721 22:14:49.338971 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0721 22:14:49.339015 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0721 22:14:49.339110 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal.example.com I0721 22:14:49.339127 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0721 22:14:49.339164 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0721 22:14:49.339244 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:49.339339 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:49.339425 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:49.339740 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.38s) === RUN TestLifecycleMinimalOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0721 22:14:49.349518 59970 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:49.419819 59970 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0721 22:14:49.590685 59970 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0721 22:14:49.600573 59970 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0721 22:14:49.639459 59970 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0721 22:14:49.643037 59970 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0721 22:14:49.646486 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.646486 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.646495 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0721 22:14:49.652355 59970 executor.go:111] Tasks: 97 done / 97 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:49.700029 59970 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0721 22:14:49.865367 59970 executor.go:111] Tasks: 0 done / 97 total; 43 can run I0721 22:14:49.868908 59970 executor.go:111] Tasks: 43 done / 97 total; 45 can run I0721 22:14:49.880485 59970 executor.go:111] Tasks: 88 done / 97 total; 5 can run I0721 22:14:49.883287 59970 executor.go:111] Tasks: 93 done / 97 total; 4 can run W0721 22:14:49.886327 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.887157 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.887984 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.888091 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.888745 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:49.888881 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0721 22:14:49.890942 59970 executor.go:111] Tasks: 97 done / 97 total; 0 can run SecurityGroup:eb46ef3b-5afd-4a48-bc7d-e7d41ed20514 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok ServerGroup:ea1eaba8-8e5c-48f1-8e55-58da2eb898d0 ok Instance:f4a70279-b6f0-4b99-b597-3c6a5a1ab464 ok SecurityGroup:497a24dc-5f3d-4f23-9eae-b73fa02c52c0 ok Instance:6890c04b-9a3f-4985-9154-efea6b5376e7 ok Instance:9cc4cedd-f075-4ab5-aab8-97142dce1efe ok ServerGroup:a0f604e0-6d3b-4a0a-b744-ffe5aa8c93ad ok SecurityGroup:79432bb7-b750-4b36-aed8-2ed6d713361e ok Network:2a21356b-8b32-4ef6-8279-f57d0a872649 ok Port:c788ed14-83dc-4054-9dd0-ba801e6213e9 ok Port:27456dbf-50cf-4a59-9989-ad07319da5e6 ok Subnet:262360b6-7dcb-4701-b371-8006192c2fdb ok Port:4eacbae9-464c-4e2a-ac99-40a1ff998a94 ok --- PASS: TestLifecycleMinimalOpenstack (1.66s) === RUN TestLifecycleMinimalGCE I0721 22:14:51.002245 59970 featureflag.go:162] FeatureFlag "AlphaAllowGCE"=true lifecycle_integration_test.go:488: running lifecycle test for cluster minimal-gce.example.com I0721 22:14:51.003928 59970 network.go:198] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0721 22:14:51.248042 59970 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0721 22:14:51.248134 59970 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0721 22:14:51.248264 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0721 22:14:51.248307 59970 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0721 22:14:51.248330 59970 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0721 22:14:51.248350 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:51.248423 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:51.248446 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:51.250028 59970 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0721 22:14:51.258237 59970 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0721 22:14:51.263443 59970 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0721 22:14:51.268130 59970 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0721 22:14:51.268401 59970 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0721 22:14:51.268515 59970 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* W0721 22:14:51.492526 59970 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0721 22:14:51.492618 59970 firewall.go:40] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0721 22:14:51.492746 59970 autoscalinggroup.go:118] enabling storage-rw for etcd backups W0721 22:14:51.492791 59970 autoscalinggroup.go:163] VMs will be configured to use the GCE default compute Service Account! This is an anti-pattern W0721 22:14:51.492814 59970 autoscalinggroup.go:164] Use a pre-created Service Account with the flag: --gce-service-account=account@projectname.iam.gserviceaccount.com I0721 22:14:51.492834 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:51.492908 59970 autoscalinggroup.go:160] VMs using Service Account: default I0721 22:14:51.492931 59970 autoscalinggroup.go:168] gsa: default I0721 22:14:51.494510 59970 executor.go:111] Tasks: 0 done / 61 total; 38 can run I0721 22:14:51.497577 59970 executor.go:111] Tasks: 38 done / 61 total; 17 can run I0721 22:14:51.502732 59970 executor.go:111] Tasks: 55 done / 61 total; 4 can run I0721 22:14:51.507337 59970 executor.go:111] Tasks: 59 done / 61 total; 2 can run I0721 22:14:51.507525 59970 executor.go:111] Tasks: 61 done / 61 total; 0 can run I0721 22:14:51.509173 59970 gce.go:93] Scanning zones: [us-test1-a] FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1658441691 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1658441691 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.51s) === RUN TestLifecycleFloatingIPOpenstack lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0721 22:14:51.519194 59970 subnets.go:182] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:51.590936 59970 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0721 22:14:51.755588 59970 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0721 22:14:51.761055 59970 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0721 22:14:51.839616 59970 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0721 22:14:51.844541 59970 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0721 22:14:51.847051 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:51.847341 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:51.847413 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0721 22:14:51.852136 59970 executor.go:111] Tasks: 99 done / 99 total; 0 can run ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:51.900829 59970 apply_cluster.go:483] Gossip DNS: skipping DNS validation I0721 22:14:52.060135 59970 executor.go:111] Tasks: 0 done / 99 total; 44 can run I0721 22:14:52.069427 59970 executor.go:111] Tasks: 44 done / 99 total; 45 can run I0721 22:14:52.077556 59970 executor.go:111] Tasks: 89 done / 99 total; 6 can run I0721 22:14:52.083052 59970 executor.go:111] Tasks: 95 done / 99 total; 4 can run W0721 22:14:52.086137 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:52.086665 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:52.087079 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:52.088018 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:52.088306 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. W0721 22:14:52.088822 59970 bootstrapscript.go:156] exporting username and password. Consider using application credentials instead. I0721 22:14:52.091426 59970 executor.go:111] Tasks: 99 done / 99 total; 0 can run Router-IF:d20e960f-000a-4dc5-87c1-58df562ba58e ok Port:c5b5211f-506c-4ef3-9971-29573af2d80f ok Instance:85937840-eab9-4371-8b66-012bab760424 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Subnet:d20e960f-000a-4dc5-87c1-58df562ba58e ok SecurityGroup:30f8f198-c401-4db7-ba11-2c8286a52524 ok ServerGroup:2edfa448-753a-4f36-9b45-b943245b1827 ok Port:8b95687b-228d-4bf5-a21a-52c5dda88138 ok Network:6569103d-75ee-4ce6-816b-9caf7b3b1c9c ok Port:b6ca23e5-40ca-4022-b81f-6799239e6e07 ok Port:a7b37180-412d-4a56-963c-cb3dec4e35c0 ok SecurityGroup:6d2a7fdf-6f96-4013-9cd9-a0036b4bd275 ok Instance:6bf5b662-8cb3-42b6-a6b1-e7b8b18ae026 ok Router:7f55311f-76b6-4c35-bca9-f2501b0c3c72 ok ServerGroup:4b01c094-6eba-47ca-98da-1628c2be830a ok Instance:73969eae-e573-494c-99db-c2f5942029fd ok SecurityGroup:a581a2c8-61a6-4075-a854-448123b66ea8 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.70s) === RUN TestLifecyclePrivateCalico I0721 22:14:53.206828 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:53.206886 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:53.206934 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:53.206989 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:53.207034 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:53.207063 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:53.207095 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:53.207132 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:53.207162 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:53.207234 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0721 22:14:53.240581 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:53.243661 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.243763 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.243828 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.283074 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:53.624397 59970 iamrole.go:91] ListRoles: { } I0721 22:14:53.626662 59970 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0721 22:14:53.627184 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:53.627347 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:53.627309 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:53.627111 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0721 22:14:53.627476 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.627618 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.627741 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0721 22:14:53.627926 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0721 22:14:53.628101 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0721 22:14:53.628140 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:53.628213 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0721 22:14:53.628285 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:53.628341 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.628505 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.628457 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" }] } I0721 22:14:53.628543 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:53.628771 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:53.628821 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0721 22:14:53.628902 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" }] } I0721 22:14:53.628856 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.629036 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:53.628994 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0721 22:14:53.629090 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:53.629148 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:53.629131 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0721 22:14:53.629236 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:53.629301 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.629428 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:53.629825 59970 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0721 22:14:53.630019 59970 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0721 22:14:53.629981 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0721 22:14:53.630156 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:53.630180 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0721 22:14:53.630309 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0721 22:14:53.630339 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0721 22:14:53.630278 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:53.630384 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:53.630496 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:53.630448 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0721 22:14:53.630543 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0721 22:14:53.630679 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:53.630755 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:53.630663 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0721 22:14:53.630802 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:53.630815 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0721 22:14:53.630901 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0721 22:14:53.630833 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.630994 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0721 22:14:53.631071 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.631877 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:53.632427 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.632769 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0721 22:14:53.632290 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:53.632854 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:53.633075 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:53.633349 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:53.633489 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:53.633609 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:53.633672 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:53.633921 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.634487 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:53.634832 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:53.634907 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:53.636557 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0721 22:14:53.636710 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0721 22:14:53.636823 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:53.636877 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.636967 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:53.637027 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:53.637078 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:53.637132 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0721 22:14:53.637192 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:53.637223 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:53.637424 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:53.637518 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:53.637587 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0721 22:14:53.637721 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:53.637810 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:53.637905 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:53.637980 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:53.638032 59970 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0721 22:14:53.638125 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:53.637832 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0721 22:14:53.639584 59970 executor.go:111] Tasks: 76 done / 119 total; 37 can run W0721 22:14:53.639819 59970 api.go:53] PageSize not implemented W0721 22:14:53.639892 59970 api.go:53] PageSize not implemented I0721 22:14:53.639796 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0721 22:14:53.640044 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0721 22:14:53.640180 59970 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0721 22:14:53.640264 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0721 22:14:53.640144 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:53.640305 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0721 22:14:53.640368 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0721 22:14:53.640459 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0721 22:14:53.640418 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0721 22:14:53.640527 59970 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0721 22:14:53.640591 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0721 22:14:53.640623 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0721 22:14:53.640670 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0721 22:14:53.640651 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0721 22:14:53.640879 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.641022 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0721 22:14:53.641199 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:53.641277 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:53.641342 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:53.641428 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:53.641533 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:53.641510 59970 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.641615 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.641761 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.641898 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.642042 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.642151 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.642244 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0721 22:14:53.642404 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:53.642554 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0721 22:14:53.642683 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" }] }] } I0721 22:14:53.642830 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0721 22:14:53.642978 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0721 22:14:53.643111 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.643238 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0721 22:14:53.643288 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0721 22:14:53.643329 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0721 22:14:53.643388 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.643505 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.643567 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.643762 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.643915 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.644058 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0721 22:14:53.644196 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:53.644318 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:53.644370 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0721 22:14:53.644468 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.644655 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0721 22:14:53.644719 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.644860 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.644959 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.645048 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.645183 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.645391 59970 executor.go:111] Tasks: 113 done / 119 total; 5 can run I0721 22:14:53.645484 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:53.645558 59970 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:53.645625 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:53.645696 59970 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0721 22:14:53.645796 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0721 22:14:53.645842 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:53.645810 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:53.645883 59970 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0721 22:14:53.645940 59970 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0721 22:14:53.646004 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:53.646128 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0721 22:14:53.646192 59970 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0721 22:14:53.646242 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:53.646277 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:53.646333 59970 dns.go:238] Pre-creating DNS records I0721 22:14:53.646358 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:53.653013 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:53.656478 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.656574 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.656638 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:53.696373 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.026599 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.028020 59970 executor.go:111] Tasks: 0 done / 119 total; 48 can run I0721 22:14:54.028245 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:54.028354 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.028683 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.028864 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.029115 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.029182 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.029394 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.029487 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:54.029512 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:54.029551 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.031244 59970 executor.go:111] Tasks: 48 done / 119 total; 28 can run I0721 22:14:54.031404 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.031582 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.031630 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.privatecalico.example.com I0721 22:14:54.031664 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.privatecalico.example.com I0721 22:14:54.031668 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:54.031699 59970 iamrole.go:163] ListAttachedRolePolicies: masters.privatecalico.example.com I0721 22:14:54.031693 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.031835 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0721 22:14:54.032121 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.032216 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0721 22:14:54.032467 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:54.032558 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0721 22:14:54.032770 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:54.032835 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.032941 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:54.032966 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0721 22:14:54.033061 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:54.033099 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.033179 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.033249 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0721 22:14:54.033336 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.033377 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.036866 59970 executor.go:111] Tasks: 76 done / 119 total; 37 can run I0721 22:14:54.036993 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0721 22:14:54.037119 59970 api.go:53] PageSize not implemented I0721 22:14:54.037141 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0721 22:14:54.037206 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W0721 22:14:54.037231 59970 api.go:53] PageSize not implemented I0721 22:14:54.037207 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.037255 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0721 22:14:54.037363 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0721 22:14:54.037406 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0721 22:14:54.037390 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.037535 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0721 22:14:54.037578 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.037849 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.038038 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.038234 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.038348 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:54.038435 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0721 22:14:54.038483 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0721 22:14:54.038743 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.038913 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.039046 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.039100 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.041255 59970 executor.go:111] Tasks: 113 done / 119 total; 5 can run W0721 22:14:54.041330 59970 api.go:53] PageSize not implemented I0721 22:14:54.041347 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0721 22:14:54.041363 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.041424 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:54.041375 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.041496 59970 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0721 22:14:54.041504 59970 api.go:53] PageSize not implemented I0721 22:14:54.041560 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0721 22:14:54.041761 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0721 22:14:54.041853 59970 executor.go:111] Tasks: 118 done / 119 total; 1 can run I0721 22:14:54.041922 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.042033 59970 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0721 22:14:54.045518 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:54.045584 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:54.045623 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:54.045659 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:54.045701 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0721 22:14:54.045741 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:54.045777 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:54.045812 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:54.045846 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:54.045885 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:54.045923 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:54.045969 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:54.046010 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0721 22:14:54.046049 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0721 22:14:54.046087 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:54.046123 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:54.046157 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:54.046196 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:54.046229 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:54.046266 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:54.046304 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:54.046340 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:54.046377 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:54.046413 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:54.046450 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.046493 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:54.046526 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:54.046565 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:54.046599 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:54.046633 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:54.046671 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:54.046711 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:54.046748 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:54.048307 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:54.048340 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:54.048382 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048452 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0721 22:14:54.048532 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048593 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048642 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048692 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048736 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048782 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0721 22:14:54.048855 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.048916 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:54.048944 59970 natgateway.go:117] DescribeNatGateways: { } I0721 22:14:54.048983 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0721 22:14:54.049027 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0721 22:14:54.049075 59970 api.go:53] PageSize not implemented I0721 22:14:54.049092 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0721 22:14:54.049147 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:54.049177 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:54.049195 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:54.049218 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:54.049235 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:54.049288 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.049317 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:54.049360 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:54.049405 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.049462 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:54.049517 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:54.049560 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0721 22:14:54.049598 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:54.049633 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.privatecalico.example.com ok I0721 22:14:54.050706 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok autoscaling-group:bastion.privatecalico.example.com ok I0721 22:14:54.050790 59970 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0721 22:14:54.050849 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.050844 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0721 22:14:54.050944 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0721 22:14:54.050847 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0721 22:14:54.051016 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0721 22:14:54.050907 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0721 22:14:54.051155 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.051128 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0721 22:14:54.051178 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:54.051190 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0721 22:14:54.051223 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0721 22:14:54.051306 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0721 22:14:54.051369 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:54.051222 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } autoscaling-config:lt-3 ok I0721 22:14:54.051440 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0721 22:14:54.051440 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:54.051533 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0721 22:14:54.051640 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.051657 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0721 22:14:54.051691 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.051741 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0721 22:14:54.051853 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.051868 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:54.051928 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0721 22:14:54.051961 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:54.052075 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0721 22:14:54.052131 59970 iamrole.go:182] ListAttachedRolePolicies: masters.privatecalico.example.com I0721 22:14:54.052108 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:54.052146 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0721 22:14:54.052182 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:54.052226 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0721 22:14:54.052229 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I0721 22:14:54.052292 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } W0721 22:14:54.052334 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.052355 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.privatecalico.example.com I0721 22:14:54.052354 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0721 22:14:54.052370 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0721 22:14:54.052402 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0721 22:14:54.052431 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0721 22:14:54.052478 59970 iamrole.go:182] ListAttachedRolePolicies: bastions.privatecalico.example.com I0721 22:14:54.052406 59970 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I0721 22:14:54.052490 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } elastic-ip:eipalloc-1 ok I0721 22:14:54.052519 59970 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0721 22:14:54.052547 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:54.052619 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0721 22:14:54.052713 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.052728 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0721 22:14:54.052791 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0721 22:14:54.052934 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0721 22:14:54.053008 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:54.053094 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:54.053184 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:54.053446 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.85s) === RUN TestLifecyclePrivateKopeio I0721 22:14:54.053982 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:54.054030 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:54.054068 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:54.054114 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:54.054159 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:54.054191 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:54.054225 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:54.054258 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:54.054284 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:54.054340 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0721 22:14:54.083204 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:54.086268 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.086361 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.086423 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.125103 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.244910 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.246859 59970 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0721 22:14:54.248549 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0721 22:14:54.248656 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0721 22:14:54.249240 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:54.249746 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0721 22:14:54.251728 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0721 22:14:54.251952 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0721 22:14:54.252070 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" }] } I0721 22:14:54.247473 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.252168 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0721 22:14:54.252730 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0721 22:14:54.252763 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] }] } I0721 22:14:54.252809 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0721 22:14:54.252823 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.252991 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.253327 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.253552 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:54.253708 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.253753 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:54.253808 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.254068 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.254292 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:54.254348 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:54.254404 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.254460 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.254597 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:54.254663 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:54.254812 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:54.254862 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:54.254949 59970 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0721 22:14:54.255091 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.255214 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0721 22:14:54.255180 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0721 22:14:54.255290 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0721 22:14:54.255348 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:54.255477 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:54.255511 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0721 22:14:54.255543 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0721 22:14:54.255642 59970 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0721 22:14:54.255650 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.255737 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0721 22:14:54.255800 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:54.255821 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0721 22:14:54.255855 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.255883 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0721 22:14:54.255956 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:54.256013 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.256059 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.256098 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0721 22:14:54.256221 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:54.256318 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.256372 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0721 22:14:54.256400 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0721 22:14:54.256504 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:54.256593 59970 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0721 22:14:54.256695 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.256832 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:54.256894 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.257084 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:54.257084 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0721 22:14:54.257316 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.257494 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:54.257499 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0721 22:14:54.257722 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.257921 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:54.257933 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.258104 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0721 22:14:54.258166 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0721 22:14:54.258252 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0721 22:14:54.258313 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.258440 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0721 22:14:54.258588 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.258633 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:54.258723 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.258901 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.259084 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0721 22:14:54.259188 59970 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0721 22:14:54.259311 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:54.259371 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:54.259412 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.259466 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:54.259508 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.259605 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:54.259658 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:54.259713 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0721 22:14:54.259761 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0721 22:14:54.259802 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:54.259848 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:54.259902 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.259942 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:54.259995 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:54.259584 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0721 22:14:54.260032 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0721 22:14:54.260074 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:54.260109 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.260194 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:54.260242 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:54.260669 59970 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0721 22:14:54.260766 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0721 22:14:54.261017 59970 api.go:53] PageSize not implemented I0721 22:14:54.260967 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0721 22:14:54.261084 59970 api.go:53] PageSize not implemented I0721 22:14:54.261086 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0721 22:14:54.261219 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.261296 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0721 22:14:54.261550 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0721 22:14:54.261552 59970 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0721 22:14:54.261757 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0721 22:14:54.261854 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0721 22:14:54.261866 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.261993 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0721 22:14:54.262168 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:54.262192 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0721 22:14:54.262255 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I0721 22:14:54.262307 59970 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] } I0721 22:14:54.262385 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.262423 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0721 22:14:54.262484 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0721 22:14:54.262567 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0721 22:14:54.262472 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" }] }] } I0721 22:14:54.262681 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.262829 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0721 22:14:54.262968 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.263138 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.263209 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0721 22:14:54.263366 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.263495 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.263761 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:54.263838 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.263971 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0721 22:14:54.264034 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0721 22:14:54.264115 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0721 22:14:54.264401 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.264588 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.264729 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:54.264867 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0721 22:14:54.265012 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.265096 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0721 22:14:54.265204 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.265368 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.265532 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:54.265645 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" }] }] } I0721 22:14:54.265790 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.265957 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.266085 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.266212 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.266336 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0721 22:14:54.266391 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.266581 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" }] }] } I0721 22:14:54.266773 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0721 22:14:54.266939 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:54.267010 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0721 22:14:54.267167 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0721 22:14:54.267237 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0721 22:14:54.267410 59970 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0721 22:14:54.267499 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.267577 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0721 22:14:54.267637 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0721 22:14:54.267698 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.267797 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.267884 59970 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0721 22:14:54.267965 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0721 22:14:54.268041 59970 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0721 22:14:54.268115 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0721 22:14:54.268191 59970 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0721 22:14:54.268274 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-4" } I0721 22:14:54.268343 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0721 22:14:54.268424 59970 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0721 22:14:54.268504 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:54.268603 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0721 22:14:54.270517 59970 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0721 22:14:54.270702 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.270699 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.270898 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.270986 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-1" } I0721 22:14:54.271068 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0721 22:14:54.271228 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-3" } I0721 22:14:54.271307 59970 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0721 22:14:54.271367 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.271400 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:54.271456 59970 dns.go:238] Pre-creating DNS records I0721 22:14:54.271485 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:54.277917 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:54.282205 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.282384 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.282532 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.321725 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.444690 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.447056 59970 executor.go:111] Tasks: 0 done / 124 total; 48 can run I0721 22:14:54.447331 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.447470 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:54.447505 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:54.447563 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.447605 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.447668 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.447733 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:54.447916 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.448174 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.448334 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.452123 59970 executor.go:111] Tasks: 48 done / 124 total; 31 can run I0721 22:14:54.452403 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0721 22:14:54.452479 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0721 22:14:54.452393 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.452995 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.453279 59970 iamrole.go:163] ListAttachedRolePolicies: masters.privatekopeio.example.com I0721 22:14:54.453248 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0721 22:14:54.454896 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.454995 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.455186 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.455298 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.455394 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.455487 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0721 22:14:54.455622 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.455784 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0721 22:14:54.455968 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.456067 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0721 22:14:54.456187 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.456283 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0721 22:14:54.456402 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.456477 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:54.456522 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:54.456584 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:54.456623 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:54.456675 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:54.456720 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:54.456781 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0721 22:14:54.456819 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.459707 59970 executor.go:111] Tasks: 79 done / 124 total; 39 can run I0721 22:14:54.459857 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.459994 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0721 22:14:54.460106 59970 api.go:53] PageSize not implemented I0721 22:14:54.460095 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.460162 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0721 22:14:54.460228 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.460327 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0721 22:14:54.460401 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } W0721 22:14:54.460456 59970 api.go:53] PageSize not implemented I0721 22:14:54.460478 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0721 22:14:54.460496 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0721 22:14:54.460616 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0721 22:14:54.460657 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0721 22:14:54.460674 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.461005 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.461323 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.461571 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0721 22:14:54.461772 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.461974 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0721 22:14:54.462178 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.462303 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.462399 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:54.465665 59970 executor.go:111] Tasks: 118 done / 124 total; 6 can run I0721 22:14:54.465758 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0721 22:14:54.465923 59970 api.go:53] PageSize not implemented I0721 22:14:54.465857 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.465918 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.465953 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0721 22:14:54.465980 59970 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0721 22:14:54.466069 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0721 22:14:54.466142 59970 api.go:53] PageSize not implemented I0721 22:14:54.466162 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0721 22:14:54.466337 59970 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0721 22:14:54.470024 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:54.470093 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:54.470134 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:54.470171 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:54.470210 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:54.470248 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:54.470281 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:54.470319 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:54.470356 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:54.470394 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:54.470432 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:54.470472 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0721 22:14:54.470515 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:54.470552 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:54.470591 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:54.470632 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:54.470668 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:54.470707 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0721 22:14:54.470745 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:54.470784 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:54.470821 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:54.470891 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:54.470930 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:54.470968 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.471008 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:54.471043 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:54.471082 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0721 22:14:54.471139 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0721 22:14:54.471181 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:54.471217 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:54.471251 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:54.471292 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:54.471333 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:54.471371 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:54.473092 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:54.473125 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:54.473166 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473231 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0721 22:14:54.473314 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473380 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473432 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473485 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473529 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473580 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0721 22:14:54.473650 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473719 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:54.473754 59970 natgateway.go:117] DescribeNatGateways: { } I0721 22:14:54.473796 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0721 22:14:54.473839 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0721 22:14:54.473888 59970 api.go:53] PageSize not implemented I0721 22:14:54.473905 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0721 22:14:54.473960 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:54.473989 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:54.474008 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:54.474032 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:54.474050 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:54.474099 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.474126 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:54.474177 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:54.474219 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.474268 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:54.474321 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:54.474372 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3","rtb-2","rtb-1"] } I0721 22:14:54.474419 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0721 22:14:54.474470 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } autoscaling-group:bastion.privatekopeio.example.com ok autoscaling-config:lt-1 ok I0721 22:14:54.475168 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0721 22:14:54.475228 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0721 22:14:54.475232 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:54.475273 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } autoscaling-group:nodes.privatekopeio.example.com ok I0721 22:14:54.475279 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0721 22:14:54.475365 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0721 22:14:54.475309 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0721 22:14:54.475436 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0721 22:14:54.475460 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0721 22:14:54.475357 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatekopeio.example.com. ok I0721 22:14:54.475529 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0721 22:14:54.475519 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } iam-instance-profile:masters.privatekopeio.example.com ok I0721 22:14:54.475499 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } load-balancer:api-privatekopeio-example-tl2bv8 ok W0721 22:14:54.475594 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.475611 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0721 22:14:54.475655 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0721 22:14:54.475792 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.475813 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0721 22:14:54.475848 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.475910 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0721 22:14:54.476107 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.476127 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0721 22:14:54.476175 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0721 22:14:54.476242 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:54.476305 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0721 22:14:54.476339 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0721 22:14:54.476462 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:54.476511 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0721 22:14:54.476593 59970 iamrole.go:182] ListAttachedRolePolicies: masters.privatekopeio.example.com I0721 22:14:54.476515 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0721 22:14:54.476627 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:54.476612 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0721 22:14:54.476653 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0721 22:14:54.476667 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0721 22:14:54.476694 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0721 22:14:54.476704 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0721 22:14:54.476750 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0721 22:14:54.476755 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0721 22:14:54.476806 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:54.476792 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0721 22:14:54.476860 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0721 22:14:54.476872 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0721 22:14:54.476910 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.476928 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0721 22:14:54.476984 59970 iamrole.go:182] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0721 22:14:54.476996 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0721 22:14:54.476959 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0721 22:14:54.477025 59970 iamrole.go:139] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } W0721 22:14:54.477036 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-role:bastions.privatekopeio.example.com ok I0721 22:14:54.477051 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:54.477133 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0721 22:14:54.477193 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:54.477250 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0721 22:14:54.477326 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:54.477407 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:54.477680 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.42s) === RUN TestLifecycleIPv6 I0721 22:14:54.478197 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:54.478247 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:54.478290 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:54.478339 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:54.478379 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:54.478409 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:54.478440 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:54.478473 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:54.478499 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:54.478551 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0721 22:14:54.503732 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:54.506991 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.507084 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.546759 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.725110 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.727319 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0721 22:14:54.727668 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.727745 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.727737 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0721 22:14:54.728017 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.728122 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0721 22:14:54.728246 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.728336 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0721 22:14:54.728336 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.728263 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:54.728450 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0721 22:14:54.728454 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.728617 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0721 22:14:54.728778 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0721 22:14:54.728771 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.729064 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:54.729318 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:54.729402 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.729682 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0721 22:14:54.729869 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:54.730098 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:54.730181 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:54.730264 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:54.730317 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:54.730366 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:54.730435 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.732111 59970 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0721 22:14:54.732265 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0721 22:14:54.732310 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.732566 59970 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0721 22:14:54.732497 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0721 22:14:54.732555 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0721 22:14:54.732673 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0721 22:14:54.732733 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0721 22:14:54.732868 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0721 22:14:54.732892 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0721 22:14:54.733126 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:54.732903 59970 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0721 22:14:54.733217 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.733434 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:54.733562 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.733980 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0721 22:14:54.734144 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0721 22:14:54.734405 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.734406 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0721 22:14:54.734648 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:54.734899 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:54.734773 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0721 22:14:54.735568 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.735657 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.735751 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.735785 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.736901 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.737045 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:54.737140 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:54.737200 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:54.737248 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:54.737292 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:54.737345 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:54.737398 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:54.737451 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.737489 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:54.737534 59970 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] } I0721 22:14:54.737580 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0721 22:14:54.740584 59970 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0721 22:14:54.740800 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0721 22:14:54.741096 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.741383 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.741698 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.741914 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.742052 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0721 22:14:54.742116 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.742269 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.742337 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.742439 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.742526 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.742532 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0721 22:14:54.742603 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.742691 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:54.742740 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.742755 59970 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.742849 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.743017 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.743227 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.743384 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0721 22:14:54.743484 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0721 22:14:54.743545 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.743678 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0721 22:14:54.743778 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0721 22:14:54.743868 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.744083 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0721 22:14:54.744325 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.744603 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.744867 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.745009 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0721 22:14:54.745153 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.745214 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:54.745302 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.745460 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0721 22:14:54.745603 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.745742 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0721 22:14:54.745950 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.746152 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:54.748315 59970 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0721 22:14:54.748446 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:54.748519 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:54.748545 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.748639 59970 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" }], Type: "network" } I0721 22:14:54.748740 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:54.748755 59970 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0721 22:14:54.748804 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0721 22:14:54.748841 59970 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0721 22:14:54.748847 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.748919 59970 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.749039 59970 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0721 22:14:54.749131 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0721 22:14:54.749180 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.749223 59970 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0721 22:14:54.749305 59970 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0721 22:14:54.749373 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.749420 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.749534 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0721 22:14:54.749625 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.749733 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.749857 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.749973 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:54.750038 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.750070 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:54.750122 59970 dns.go:238] Pre-creating DNS records I0721 22:14:54.750146 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:54.755844 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:54.759074 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.759168 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.798284 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:54.975966 59970 iamrole.go:91] ListRoles: { } I0721 22:14:54.978204 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0721 22:14:54.978501 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:54.978743 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.978949 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.979093 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:54.979245 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.979461 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.979652 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:54.979723 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:54.979795 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.979852 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I0721 22:14:54.981514 59970 executor.go:111] Tasks: 46 done / 105 total; 21 can run I0721 22:14:54.981681 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.981748 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0721 22:14:54.981866 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0721 22:14:54.981821 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.981926 59970 iamrole.go:163] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0721 22:14:54.981879 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0721 22:14:54.981986 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0721 22:14:54.982155 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:54.982204 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0721 22:14:54.982339 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:54.982388 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.982483 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:54.982559 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.982690 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0721 22:14:54.982836 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:54.986694 59970 executor.go:111] Tasks: 67 done / 105 total; 30 can run I0721 22:14:54.986923 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.987025 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:54.987093 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.987242 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.987340 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.987587 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.987749 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:54.987846 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0721 22:14:54.987898 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0721 22:14:54.988095 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.988242 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:54.988415 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.988600 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.988827 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:54.991334 59970 executor.go:111] Tasks: 97 done / 105 total; 5 can run I0721 22:14:54.991428 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.991534 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.991635 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:54.991606 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:54.991702 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:54.991723 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0721 22:14:54.991779 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0721 22:14:54.991813 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0721 22:14:54.993061 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0721 22:14:54.993106 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0721 22:14:54.993264 59970 executor.go:111] Tasks: 102 done / 105 total; 3 can run I0721 22:14:54.993362 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.993381 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:54.993446 59970 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0721 22:14:54.993453 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:54.993530 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:54.993546 59970 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0721 22:14:54.993562 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0721 22:14:54.993634 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:54.993662 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0721 22:14:54.993760 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:54.996801 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:54.996865 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:54.996902 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:54.996938 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:54.996974 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0721 22:14:54.997010 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:54.997044 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:54.997077 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:54.997112 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:54.997150 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:54.997186 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:54.997226 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:54.997270 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0721 22:14:54.997308 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:54.997343 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:54.997378 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:54.997416 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:54.997450 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:54.997488 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:54.997525 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:54.997561 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:54.997596 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:54.997634 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:54.997668 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:54.997703 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:54.997738 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:54.997798 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:54.997841 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:54.997878 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:54.999434 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:54.999466 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:54.999507 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999564 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0721 22:14:54.999629 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999689 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999740 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999792 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999832 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999879 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0721 22:14:54.999933 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:54.999979 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:55.000006 59970 natgateway.go:117] DescribeNatGateways: { } I0721 22:14:55.000048 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0721 22:14:55.000091 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0721 22:14:55.000144 59970 api.go:53] PageSize not implemented I0721 22:14:55.000164 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:55.000192 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:55.000206 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0721 22:14:55.000245 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:55.000268 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:55.000284 59970 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0721 22:14:55.000342 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.000369 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.000418 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:55.000450 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.000492 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:55.000542 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:55.000587 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0721 22:14:55.000628 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:55.000664 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0721 22:14:55.001224 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.001380 59970 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I0721 22:14:55.001406 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } autoscaling-group:nodes.minimal-ipv6.example.com ok I0721 22:14:55.001503 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0721 22:14:55.001498 59970 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0721 22:14:55.001316 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.minimal-ipv6.example.com ok I0721 22:14:55.001575 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0721 22:14:55.001672 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I0721 22:14:55.001292 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0721 22:14:55.001728 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:55.001760 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:55.001822 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:55.001901 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0721 22:14:55.001950 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0721 22:14:55.002018 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok autoscaling-config:lt-1 ok I0721 22:14:55.002097 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:55.002158 59970 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0721 22:14:55.002211 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.002262 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0721 22:14:55.002407 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:55.002428 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0721 22:14:55.002506 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0721 22:14:55.002587 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:55.002649 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0721 22:14:55.002736 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0721 22:14:55.002788 59970 iamrole.go:182] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0721 22:14:55.002802 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0721 22:14:55.002835 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0721 22:14:55.002804 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:55.002889 59970 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0721 22:14:55.002883 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0721 22:14:55.002973 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0721 22:14:55.002991 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0721 22:14:55.003024 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0721 22:14:55.003118 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:55.003210 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:55.003301 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:55.003589 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.53s) === RUN TestLifecycleSharedVPC I0721 22:14:55.004188 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:55.004236 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:55.004277 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:55.004323 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:55.004363 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:55.004391 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:55.004424 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:55.004485 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:55.004510 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:55.004569 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0721 22:14:55.032654 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:55.035832 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.035961 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.074871 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.177353 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.182671 59970 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0721 22:14:55.183051 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0721 22:14:55.183145 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] }] } I0721 22:14:55.183287 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:55.183370 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:55.183230 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I0721 22:14:55.183354 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:55.183490 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.183608 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I0721 22:14:55.183695 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:55.183844 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:55.183823 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0721 22:14:55.183903 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:55.183985 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:55.184011 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0721 22:14:55.184016 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.184073 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0721 22:14:55.184129 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:55.184183 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:55.184202 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:55.185584 59970 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0721 22:14:55.185766 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0721 22:14:55.185805 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] }] } I0721 22:14:55.185945 59970 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0721 22:14:55.185969 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0721 22:14:55.185988 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:55.185998 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I0721 22:14:55.186071 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0721 22:14:55.186113 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.186233 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:55.186307 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0721 22:14:55.186361 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0721 22:14:55.186465 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:55.186518 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" } ] } I0721 22:14:55.186607 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:55.186726 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:55.186795 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.186752 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0721 22:14:55.186852 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:55.186928 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:55.186974 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0721 22:14:55.187204 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:55.187334 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:55.187423 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.188434 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0721 22:14:55.191366 59970 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0721 22:14:55.191592 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0721 22:14:55.191760 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.191862 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.192034 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:55.192084 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0721 22:14:55.192214 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I0721 22:14:55.192383 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0721 22:14:55.192517 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0721 22:14:55.192738 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.192898 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.192971 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.193116 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.193312 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0721 22:14:55.193391 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.193629 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.193827 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0721 22:14:55.194057 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.194196 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.194333 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0721 22:14:55.194490 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" }] }] } I0721 22:14:55.194639 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" }] }] } I0721 22:14:55.194768 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0721 22:14:55.194897 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.195007 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.195081 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0721 22:14:55.197311 59970 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0721 22:14:55.197602 59970 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0721 22:14:55.197663 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.197699 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.197760 59970 dns.go:238] Pre-creating DNS records I0721 22:14:55.197809 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.203404 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:55.206204 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.206306 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.246388 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.349815 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.351280 59970 executor.go:111] Tasks: 0 done / 83 total; 43 can run I0721 22:14:55.351558 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:55.351646 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.351811 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:55.351899 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:55.352055 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.352215 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:55.352322 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:55.352364 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:55.356136 59970 executor.go:111] Tasks: 43 done / 83 total; 17 can run I0721 22:14:55.356409 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:55.356598 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0721 22:14:55.356787 59970 iamrole.go:163] ListAttachedRolePolicies: masters.sharedvpc.example.com I0721 22:14:55.356705 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.359583 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.359881 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:55.359955 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0721 22:14:55.360204 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0721 22:14:55.360493 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.360611 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.364564 59970 executor.go:111] Tasks: 60 done / 83 total; 21 can run I0721 22:14:55.364794 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.364943 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.365154 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.365347 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.365487 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.365667 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.366001 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.368335 59970 executor.go:111] Tasks: 81 done / 83 total; 2 can run I0721 22:14:55.368573 59970 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0721 22:14:55.371750 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:55.371814 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:55.371850 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:55.371900 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:55.371941 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:55.371989 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:55.372027 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:55.372084 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:55.372137 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:55.372194 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:55.372234 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:55.372281 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:55.372321 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:55.372367 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:55.372407 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:55.372461 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:55.372513 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:55.372559 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:55.372596 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:55.372638 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:55.372682 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:55.372729 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:55.374380 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:55.374432 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:55.374535 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.374664 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0721 22:14:55.374796 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.374935 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.375036 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.375124 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.375178 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.375216 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0721 22:14:55.375260 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.375316 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0721 22:14:55.375353 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0721 22:14:55.375397 59970 api.go:53] PageSize not implemented I0721 22:14:55.375418 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:55.375446 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:55.375463 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:55.375487 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:55.375505 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:55.375562 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.375592 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.375635 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:55.375695 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.375738 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:55.375778 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:55.375807 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0721 22:14:55.376219 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } autoscaling-group:nodes.sharedvpc.example.com ok I0721 22:14:55.376319 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0721 22:14:55.376276 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:nodes.sharedvpc.example.com ok volume:vol-2 ok I0721 22:14:55.376370 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0721 22:14:55.376399 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.376451 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0721 22:14:55.376275 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0721 22:14:55.376486 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0721 22:14:55.376646 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:55.376669 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:55.376712 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0721 22:14:55.376833 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:55.376937 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.377000 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0721 22:14:55.377218 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:55.377234 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:55.377329 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0721 22:14:55.377400 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:55.377385 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0721 22:14:55.377484 59970 iamrole.go:182] ListAttachedRolePolicies: masters.sharedvpc.example.com I0721 22:14:55.377515 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0721 22:14:55.377572 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0721 22:14:55.377643 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0721 22:14:55.377695 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0721 22:14:55.377812 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0721 22:14:55.378004 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:55.378378 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.37s) === RUN TestLifecycleComplex I0721 22:14:55.378888 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:55.378939 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:55.378980 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:55.379024 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:55.379067 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:55.379097 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:55.379124 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:55.379157 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:55.379184 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:55.379240 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0721 22:14:55.412818 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:55.421001 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.421189 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.460436 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.569311 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.571584 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0721 22:14:55.571813 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.572091 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } W0721 22:14:55.572305 59970 api.go:53] PageSize not implemented I0721 22:14:55.572069 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0721 22:14:55.572313 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:55.572462 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:55.572506 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0721 22:14:55.572715 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0721 22:14:55.572816 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0721 22:14:55.572757 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:55.572910 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0721 22:14:55.572993 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:55.572986 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] } I0721 22:14:55.573076 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:55.573264 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:55.573342 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:55.573162 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0721 22:14:55.573381 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:55.573415 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:55.573412 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", PermissionsBoundary: "arn:aws-test:iam::000000000000:policy/boundaries", RoleName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0721 22:14:55.573460 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.573568 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:55.573645 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:55.573706 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:55.573774 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.573812 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0721 22:14:55.573941 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:55.574811 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:55.574939 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:55.574963 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.575083 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:55.575237 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:55.575154 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0721 22:14:55.575156 59970 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0721 22:14:55.575230 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.575367 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:55.575399 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0721 22:14:55.575503 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:55.575399 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.575601 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0721 22:14:55.575677 59970 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0721 22:14:55.575604 59970 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0721 22:14:55.575730 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.575864 59970 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0721 22:14:55.575910 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:55.576043 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0721 22:14:55.576073 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.576225 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0721 22:14:55.576335 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:55.576391 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.576493 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.576593 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:55.576726 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.576825 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:55.576925 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.576997 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0721 22:14:55.576981 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0721 22:14:55.577081 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:55.577216 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0721 22:14:55.577153 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0721 22:14:55.577314 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0721 22:14:55.577374 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:55.577395 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0721 22:14:55.577420 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0721 22:14:55.577558 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:55.577639 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.577299 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0721 22:14:55.577702 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:55.577808 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.577869 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:55.577950 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:55.578004 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0721 22:14:55.578134 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:55.578219 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:55.578321 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:55.578383 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:55.578429 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:55.578468 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0721 22:14:55.578596 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:55.578646 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" } ] } I0721 22:14:55.578784 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:55.578834 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:55.578880 59970 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] } I0721 22:14:55.579926 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:55.580115 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0721 22:14:55.580229 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0721 22:14:55.580249 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0721 22:14:55.580315 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:55.580361 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.580531 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:55.580548 59970 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-3" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0721 22:14:55.580701 59970 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0721 22:14:55.580672 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" } ] }] } I0721 22:14:55.580791 59970 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0721 22:14:55.580870 59970 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [ { Key: "load_balancing.cross_zone.enabled", Value: "true" }, { Key: "access_logs.s3.enabled", Value: "true" }, { Key: "access_logs.s3.bucket", Value: "access-log-example" }, { Key: "access_logs.s3.prefix", Value: "" } ], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:55.580929 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" } ] }] } I0721 22:14:55.581195 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0721 22:14:55.581305 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.581419 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.581478 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.581605 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.581862 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0721 22:14:55.582138 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0721 22:14:55.582346 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.582490 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.582607 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:55.582758 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.582870 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0721 22:14:55.583008 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0721 22:14:55.583160 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0721 22:14:55.583336 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0721 22:14:55.583509 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0721 22:14:55.583666 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I0721 22:14:55.583814 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0721 22:14:55.583858 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0721 22:14:55.584000 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0721 22:14:55.584096 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I0721 22:14:55.584262 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:55.584311 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0721 22:14:55.584403 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" } ] }] } I0721 22:14:55.584585 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0721 22:14:55.584678 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:55.584720 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0721 22:14:55.584868 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0721 22:14:55.584957 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0721 22:14:55.585053 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0721 22:14:55.585252 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0721 22:14:55.585399 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0721 22:14:55.585547 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0721 22:14:55.585706 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0721 22:14:55.585751 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0721 22:14:55.585842 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0721 22:14:55.585983 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0721 22:14:55.586103 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0721 22:14:55.586142 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0721 22:14:55.586190 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0721 22:14:55.586250 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:55.586391 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:55.586560 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:55.586683 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0721 22:14:55.586735 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.586766 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.586819 59970 dns.go:238] Pre-creating DNS records I0721 22:14:55.586842 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.599330 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:55.603481 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.603670 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.642803 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.745842 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.747698 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0721 22:14:55.747931 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0721 22:14:55.748855 59970 api.go:53] PageSize not implemented I0721 22:14:55.748035 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:55.750027 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.750240 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:55.750367 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.750542 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.750684 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:55.750764 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.750884 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:55.750990 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:55.751224 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:55.751406 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.755528 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:55.755724 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:55.755702 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.755923 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:55.755868 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.755957 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:55.755996 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.756085 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:55.756175 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:55.756226 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:55.756233 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:55.756354 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:55.756265 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.756460 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.756629 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:55.756671 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:55.756870 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:55.756947 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:55.757160 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.757213 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.757328 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:55.757488 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.757585 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.757714 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.757822 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.757953 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.760883 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:55.761087 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:55.761241 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:55.761274 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.761345 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.761186 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:55.761395 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:55.761454 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:55.761491 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:55.761526 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:55.761518 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.761838 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.761965 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.762256 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:55.762444 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.762653 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.762746 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.763041 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.763184 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.765874 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:55.765980 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:55.766088 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:55.766114 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:55.766137 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.766207 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0721 22:14:55.772789 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.776145 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.787805 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:55.791517 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.791614 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.831756 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.940587 59970 iamrole.go:91] ListRoles: { } I0721 22:14:55.942866 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0721 22:14:55.943075 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:55.943201 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:55.943358 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.943387 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0721 22:14:55.943700 59970 api.go:53] PageSize not implemented I0721 22:14:55.943567 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.943826 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.943980 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:55.944012 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:55.944062 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:55.944249 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.944463 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:55.944584 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.946370 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:55.946525 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:55.946560 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.946739 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:55.946689 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:55.946744 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.946803 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:55.946930 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.946964 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:55.947056 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:55.947029 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.947167 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:55.947321 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:55.947382 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.947528 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:55.947571 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:55.947731 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:55.947876 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.948044 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.948193 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.948419 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:55.948493 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:55.948677 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:55.948724 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:55.948838 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:55.948940 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:55.952248 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:55.952575 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.952688 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.952692 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:55.952811 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:55.952841 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.952836 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:55.952908 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.952961 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:55.953011 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:55.953063 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:55.953031 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.953108 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:55.953217 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.953329 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0721 22:14:55.953380 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:55.953424 59970 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0721 22:14:55.953467 59970 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0721 22:14:55.953438 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.953508 59970 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0721 22:14:55.953565 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.953626 59970 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0721 22:14:55.953711 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.953757 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.953753 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.953881 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:55.954073 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:55.960601 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:55.960719 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:55.960819 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:55.960844 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:55.960867 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:55.960942 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0721 22:14:55.961010 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:55.961044 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:55.961097 59970 dns.go:238] Pre-creating DNS records I0721 22:14:55.969000 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:55.973411 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:55.973514 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.013403 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.109819 59970 iamrole.go:91] ListRoles: { } I0721 22:14:56.112048 59970 executor.go:111] Tasks: 0 done / 116 total; 48 can run I0721 22:14:56.112259 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.112394 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:56.112430 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W0721 22:14:56.112449 59970 api.go:53] PageSize not implemented I0721 22:14:56.112490 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.112533 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:56.112645 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:56.112684 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:56.112756 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.112945 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.113287 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.113516 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:56.113712 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:56.114829 59970 executor.go:111] Tasks: 48 done / 116 total; 27 can run I0721 22:14:56.115015 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:56.115097 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.115025 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:56.115175 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:56.115320 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:56.115354 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:56.115270 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:56.115414 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.115382 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:56.115404 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:56.115679 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:56.115841 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.115991 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:56.116070 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.116212 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.116353 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:56.116399 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.116488 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.116575 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.116655 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.116786 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:56.116837 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.117047 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:56.117096 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.117241 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0721 22:14:56.119708 59970 executor.go:111] Tasks: 75 done / 116 total; 38 can run I0721 22:14:56.119948 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.120132 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.120222 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.120108 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:56.120342 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:56.120370 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.120370 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.120479 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.120525 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.120568 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.120606 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.120639 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.120639 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.120834 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.120923 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.121206 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.121387 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.121628 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.124074 59970 executor.go:111] Tasks: 113 done / 116 total; 3 can run I0721 22:14:56.124324 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:56.124426 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:56.124474 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:56.124503 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.124591 59970 executor.go:111] Tasks: 116 done / 116 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0721 22:14:56.128221 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.131522 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.147210 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:56.150944 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.151051 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.190264 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.297619 59970 iamrole.go:91] ListRoles: { } I0721 22:14:56.299940 59970 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0721 22:14:56.300538 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:56.300738 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.300890 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0721 22:14:56.300946 59970 api.go:53] PageSize not implemented I0721 22:14:56.300994 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:56.300927 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:56.301067 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0721 22:14:56.301223 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.301470 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:56.301573 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.301678 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.301759 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:56.301780 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:56.301824 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.303451 59970 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0721 22:14:56.303607 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:56.303708 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.303702 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0721 22:14:56.303901 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:56.303860 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.304004 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.304073 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:56.304058 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:56.304142 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.304103 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:56.304329 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:56.304399 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:56.304467 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:56.304636 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:56.304683 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0721 22:14:56.304881 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:56.304927 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:56.305087 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:56.305140 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.305285 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.305432 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.305576 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:56.305619 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0721 22:14:56.305759 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.305850 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.309757 59970 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0721 22:14:56.309870 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.310143 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:56.310257 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:56.310202 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.310284 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.310355 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.310363 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.310415 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.310478 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.310541 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.310597 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.310644 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.310808 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.310954 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.311175 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.311251 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.311720 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.312066 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.313951 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.314026 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.316331 59970 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0721 22:14:56.316416 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:56.316536 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:56.316571 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:56.316605 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.316747 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0721 22:14:56.316808 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.316844 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:56.316900 59970 dns.go:238] Pre-creating DNS records I0721 22:14:56.324308 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:56.328045 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.328145 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.368442 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.472723 59970 iamrole.go:91] ListRoles: { } I0721 22:14:56.475071 59970 executor.go:111] Tasks: 0 done / 117 total; 49 can run W0721 22:14:56.475572 59970 api.go:53] PageSize not implemented I0721 22:14:56.475550 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:56.475722 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:56.475992 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0721 22:14:56.475988 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:56.476130 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:56.476236 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.476378 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:56.476413 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:56.476451 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.476704 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.476956 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:56.477133 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.477359 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.478225 59970 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0721 22:14:56.478431 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:56.478605 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:56.478664 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:56.478707 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:56.478635 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:56.478817 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.478909 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:56.478891 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:56.478980 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:56.478980 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.479066 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.479229 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:56.479274 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:56.479443 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:56.479601 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.479807 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.479913 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.480049 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.480174 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.480268 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.480351 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.480501 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:56.480543 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.480706 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:56.480759 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.483845 59970 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0721 22:14:56.484096 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.484287 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:56.484379 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:56.484284 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.484399 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.484474 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.484516 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.484557 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.484602 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.484638 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.484549 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.484839 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.484955 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.485121 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.485226 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.485294 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.485431 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.485502 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.491068 59970 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0721 22:14:56.491266 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:56.491379 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:56.491404 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:56.491430 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.491520 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0721 22:14:56.496682 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.500493 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.500585 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.509732 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:56.513456 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.513551 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.513612 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.552887 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.653028 59970 iamrole.go:91] ListRoles: { } I0721 22:14:56.654867 59970 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0721 22:14:56.655105 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.657482 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:56.657554 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:56.657684 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.658032 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0721 22:14:56.658080 59970 api.go:53] PageSize not implemented I0721 22:14:56.658153 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.658342 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:56.658529 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:56.658620 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0721 22:14:56.658764 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.658911 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:56.655249 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:56.659185 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.662892 59970 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0721 22:14:56.663011 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.663157 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:56.663246 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:56.663277 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:56.663287 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.663377 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:56.663382 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.663455 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.663472 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:56.663635 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:56.663621 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.663682 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.663770 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:56.663928 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:56.663984 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.664152 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.664307 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.664569 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:56.664641 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.664811 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.665001 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.665188 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:56.665442 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:56.665556 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:56.665674 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:56.668780 59970 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0721 22:14:56.669103 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.669117 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:56.669200 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:56.669224 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.669295 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.669335 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.669284 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.669377 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.669410 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.669450 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.669516 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.669658 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.669769 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.669922 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.670126 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.670339 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.670873 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.671081 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.673516 59970 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0721 22:14:56.673621 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:56.673729 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:56.673761 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:56.673806 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.673903 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0721 22:14:56.673985 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.674023 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:56.674091 59970 dns.go:238] Pre-creating DNS records I0721 22:14:56.682129 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:56.686112 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.686216 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.686301 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.726343 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.836301 59970 iamrole.go:91] ListRoles: { } I0721 22:14:56.838594 59970 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0721 22:14:56.838820 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0721 22:14:56.838895 59970 api.go:53] PageSize not implemented I0721 22:14:56.839009 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.839159 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:56.839231 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0721 22:14:56.839391 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:56.839591 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.839802 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.839941 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:56.840044 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:56.840141 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.840255 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:56.840289 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:56.840349 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.841648 59970 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0721 22:14:56.841797 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.841826 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:56.841994 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.841941 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.842191 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:56.842135 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:56.842252 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:56.842169 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:56.842292 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.842376 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:56.842451 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:56.842636 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:56.842734 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:56.842816 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.842974 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.843078 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.843210 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:56.843258 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:56.843456 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:56.843510 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.843606 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.843711 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0721 22:14:56.843825 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:56.843921 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:56.844004 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:56.848451 59970 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0721 22:14:56.848660 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.848852 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.849055 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:56.849180 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:56.849208 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.849402 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.849232 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.849574 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.849755 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:56.850172 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:56.850268 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:56.850137 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:56.850830 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.851058 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.851985 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.852175 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:56.852285 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.852432 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.854396 59970 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0721 22:14:56.854644 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:56.854748 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:56.854776 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:56.854805 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:56.854901 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0721 22:14:56.858249 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:56.861985 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.862080 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.862138 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.872192 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:56.875763 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.875850 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.875906 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.875955 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:56.916122 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.025509 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.027794 59970 executor.go:111] Tasks: 0 done / 117 total; 49 can run W0721 22:14:57.028135 59970 api.go:53] PageSize not implemented I0721 22:14:57.028072 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:57.028292 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.028476 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0721 22:14:57.028573 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:57.028741 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:57.028745 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:57.028898 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.029077 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:57.029125 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:57.029198 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.029444 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.029584 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:57.029700 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.030957 59970 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0721 22:14:57.031128 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.031348 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.031420 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:57.031485 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:57.031469 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:57.031471 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:57.031613 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:57.031701 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:57.031686 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:57.031778 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:57.031771 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:57.031966 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.032024 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0721 22:14:57.032184 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:57.032230 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.032366 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.032504 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:57.032557 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.032712 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:57.032771 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.032863 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:57.033021 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.033079 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.033188 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.033259 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.036298 59970 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0721 22:14:57.036482 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.036671 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.036748 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.036777 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:57.036956 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:57.036980 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.037037 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.037075 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:57.037113 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:57.037148 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:57.037178 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:57.037163 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.037343 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.037547 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.037767 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.037883 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:57.038148 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.038382 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:57.038964 59970 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0721 22:14:57.039058 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:57.039143 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:57.039166 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:57.039191 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.039379 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0721 22:14:57.039446 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.039482 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.039548 59970 dns.go:238] Pre-creating DNS records I0721 22:14:57.047109 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:57.051261 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.051364 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.051427 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.051481 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.090694 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.195700 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.197985 59970 executor.go:111] Tasks: 0 done / 117 total; 49 can run I0721 22:14:57.198229 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0721 22:14:57.198374 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:57.198391 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0721 22:14:57.198516 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0721 22:14:57.198717 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:57.198786 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W0721 22:14:57.198751 59970 api.go:53] PageSize not implemented I0721 22:14:57.198869 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.199160 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0721 22:14:57.199347 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0721 22:14:57.199506 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.199756 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.199979 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.200057 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0721 22:14:57.201330 59970 executor.go:111] Tasks: 49 done / 117 total; 27 can run I0721 22:14:57.201512 59970 iamrole.go:163] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:57.201518 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.201632 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0721 22:14:57.201717 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:57.201697 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.201831 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:57.201859 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0721 22:14:57.201817 59970 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0721 22:14:57.201905 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.201965 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:57.202019 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0721 22:14:57.202242 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.202304 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:57.202354 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.202448 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.202582 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.202672 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0721 22:14:57.202817 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.202886 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0721 22:14:57.203097 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:57.203166 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.203286 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.203437 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.203550 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0721 22:14:57.203731 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:57.208106 59970 executor.go:111] Tasks: 76 done / 117 total; 38 can run I0721 22:14:57.208450 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.208554 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.208543 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:57.208695 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:57.208723 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.208786 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.208825 59970 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:57.208867 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I0721 22:14:57.208904 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I0721 22:14:57.208934 59970 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:57.208936 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:57.209267 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:57.209369 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.209600 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.209713 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.209786 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.209983 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.211024 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.216148 59970 executor.go:111] Tasks: 114 done / 117 total; 3 can run I0721 22:14:57.216322 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:57.216431 59970 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0721 22:14:57.216456 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0721 22:14:57.216481 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.216567 59970 executor.go:111] Tasks: 117 done / 117 total; 0 can run I0721 22:14:57.216911 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:57.216968 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:57.217004 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:57.217037 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:57.217075 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:57.217110 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:57.217143 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:57.217176 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:57.217213 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:57.217255 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:57.217299 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:57.217332 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:57.217364 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:57.217402 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:57.217435 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:57.217472 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:57.217508 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:57.217545 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:57.217582 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:57.217619 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:57.217654 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:57.217692 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0721 22:14:57.217729 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:57.217762 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:57.217831 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:57.217888 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:57.217942 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:57.217988 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:57.220617 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:57.220647 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:57.220681 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.220741 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0721 22:14:57.220808 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.220872 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.220922 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.220980 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.221023 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.221069 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0721 22:14:57.221147 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.221225 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0721 22:14:57.221267 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0721 22:14:57.221317 59970 api.go:53] PageSize not implemented I0721 22:14:57.221334 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:57.221358 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:57.221372 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0721 22:14:57.221409 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:57.221432 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:57.221449 59970 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2" ] } I0721 22:14:57.221517 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.221543 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.221584 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:57.221620 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.221662 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:57.221714 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:57.221763 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0721 22:14:57.222323 59970 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok I0721 22:14:57.222395 59970 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0721 22:14:57.222378 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0721 22:14:57.222461 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-group:nodes.complex.example.com ok I0721 22:14:57.222466 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok autoscaling-group:master-us-test-1a.masters.complex.example.com ok volume:vol-2 ok autoscaling-config:lt-1 ok I0721 22:14:57.222601 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0721 22:14:57.222628 59970 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I0721 22:14:57.222676 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0721 22:14:57.222770 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0721 22:14:57.222458 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.222522 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 } ] } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok W0721 22:14:57.222869 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:57.222887 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0721 22:14:57.222922 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:57.222994 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok autoscaling-config:lt-2 ok I0721 22:14:57.223052 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:57.223118 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0721 22:14:57.223151 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.223206 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:57.223246 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0721 22:14:57.223290 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.223359 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W0721 22:14:57.223671 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:57.223683 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:57.223773 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0721 22:14:57.223824 59970 iamrole.go:182] ListAttachedRolePolicies: masters.complex.example.com I0721 22:14:57.223791 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-3" } I0721 22:14:57.223838 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } subnet:subnet-3 ok I0721 22:14:57.223868 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0721 22:14:57.223901 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0721 22:14:57.223940 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.complex.example.com I0721 22:14:57.223905 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } I0721 22:14:57.223951 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } route-table:rtb-2 ok I0721 22:14:57.223976 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0721 22:14:57.224046 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:57.224149 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:57.224272 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:57.224614 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (1.85s) === RUN TestLifecycleExternalLB I0721 22:14:57.225146 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:57.225194 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:57.225236 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:57.225285 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:57.225327 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:57.225356 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:57.225387 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:57.225418 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:57.225443 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:57.225497 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0721 22:14:57.250726 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:57.253553 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.253651 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.294489 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.402334 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.404043 59970 executor.go:111] Tasks: 0 done / 93 total; 51 can run I0721 22:14:57.404536 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.404711 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:57.404655 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0721 22:14:57.404702 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0721 22:14:57.404881 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.404891 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0721 22:14:57.404909 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0721 22:14:57.404803 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } W0721 22:14:57.405149 59970 api.go:53] PageSize not implemented I0721 22:14:57.404964 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0721 22:14:57.405265 59970 api.go:53] PageSize not implemented I0721 22:14:57.404970 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.405194 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } W0721 22:14:57.405405 59970 api.go:53] PageSize not implemented I0721 22:14:57.405246 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.405431 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.405530 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.externallb.example.com", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] } I0721 22:14:57.405639 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:57.405614 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.405775 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:57.405813 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.405884 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.405745 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0721 22:14:57.405893 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.405971 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0721 22:14:57.406111 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.406339 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:57.406383 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:57.406423 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:57.406475 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.406514 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:57.406657 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:57.406794 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.406908 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:57.406953 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:57.407009 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:57.407076 59970 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0721 22:14:57.407255 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0721 22:14:57.407311 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0721 22:14:57.407304 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.407485 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0721 22:14:57.407442 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0721 22:14:57.407567 59970 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0721 22:14:57.407611 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:57.407549 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0721 22:14:57.407715 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:57.407838 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:57.407764 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:57.407914 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0721 22:14:57.408141 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0721 22:14:57.408208 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" }] }], VpcId: "vpc-1" } I0721 22:14:57.408356 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:57.408409 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.408453 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.408647 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0721 22:14:57.408747 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0721 22:14:57.408870 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.408945 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.409037 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:57.409089 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:57.409137 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:57.409222 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0721 22:14:57.409320 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:57.409360 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:57.409451 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:57.409513 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.409483 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0721 22:14:57.412005 59970 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0721 22:14:57.412151 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.412263 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.412416 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:57.412473 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0721 22:14:57.412602 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.412722 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.412877 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.413052 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.413278 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0721 22:14:57.413426 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.413580 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.413664 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:57.413723 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.414933 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.415222 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.415548 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.415761 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.416147 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I0721 22:14:57.416432 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.416769 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:57.417098 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.419683 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.419896 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.420054 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0721 22:14:57.420201 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:57.421396 59970 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0721 22:14:57.421668 59970 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0721 22:14:57.421733 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.421771 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.421834 59970 dns.go:238] Pre-creating DNS records I0721 22:14:57.421861 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.428490 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:57.431763 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.431870 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.471811 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.573072 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.574702 59970 executor.go:111] Tasks: 0 done / 93 total; 51 can run W0721 22:14:57.574871 59970 api.go:53] PageSize not implemented I0721 22:14:57.574945 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:57.574997 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0721 22:14:57.575084 59970 api.go:53] PageSize not implemented I0721 22:14:57.575028 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:57.575132 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.576061 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.576261 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:57.575171 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.576396 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0721 22:14:57.576466 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0721 22:14:57.576755 59970 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.576800 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:57.576853 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0721 22:14:57.575220 59970 api.go:53] PageSize not implemented I0721 22:14:57.577887 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.578031 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.578307 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.578482 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:57.578520 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:57.579253 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.581439 59970 executor.go:111] Tasks: 51 done / 93 total; 19 can run I0721 22:14:57.581579 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.externallb.example.com I0721 22:14:57.581652 59970 iamrole.go:163] ListAttachedRolePolicies: masters.externallb.example.com I0721 22:14:57.581706 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0721 22:14:57.581879 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.581973 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] }] } I0721 22:14:57.582149 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:57.582209 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0721 22:14:57.582381 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.582445 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.582622 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0721 22:14:57.582834 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.583050 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:57.586915 59970 executor.go:111] Tasks: 70 done / 93 total; 21 can run I0721 22:14:57.587080 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.587255 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.587367 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.587640 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.587747 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.587888 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.588044 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.590291 59970 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0721 22:14:57.590606 59970 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0721 22:14:57.593822 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:57.593886 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:57.593924 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:57.593956 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:57.593997 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:57.594035 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:57.594069 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:57.594104 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:57.594138 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:57.594172 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:57.594208 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:57.594249 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:57.594280 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:57.594311 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:57.594344 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:57.594375 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:57.594410 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:57.594464 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:57.594504 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:57.594536 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:57.594568 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:57.594599 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:57.594635 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:57.594669 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:57.594704 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:57.596078 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:57.596103 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:57.596140 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596192 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0721 22:14:57.596248 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596304 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596350 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596409 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596450 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596489 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0721 22:14:57.596534 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596588 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0721 22:14:57.596628 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0721 22:14:57.596674 59970 api.go:53] PageSize not implemented I0721 22:14:57.596694 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:57.596722 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:57.596741 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:57.596764 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:57.596779 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0721 22:14:57.596828 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.596856 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.596905 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:57.596941 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.596984 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:57.597027 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:57.597061 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:57.597518 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.externallb.example.com ok autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0721 22:14:57.597592 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } keypair:key-1 ok I0721 22:14:57.597645 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0721 22:14:57.597731 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0721 22:14:57.597682 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0721 22:14:57.597797 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } W0721 22:14:57.597843 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:57.597848 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0721 22:14:57.597871 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:57.597699 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0721 22:14:57.597926 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0721 22:14:57.597996 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:57.598061 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0721 22:14:57.598104 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.598163 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0721 22:14:57.598382 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:57.598403 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:57.598454 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0721 22:14:57.598550 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0721 22:14:57.598602 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.externallb.example.com I0721 22:14:57.598578 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:57.598617 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0721 22:14:57.598681 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0721 22:14:57.598739 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0721 22:14:57.598793 59970 iamrole.go:182] ListAttachedRolePolicies: masters.externallb.example.com I0721 22:14:57.598809 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0721 22:14:57.598841 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0721 22:14:57.598924 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:57.599011 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:57.599092 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:57.599377 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.37s) === RUN TestLifecycleSharedSubnet I0721 22:14:57.600022 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:57.600072 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:57.600110 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:57.600159 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:57.600201 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:57.600230 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:57.600258 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:57.600288 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:57.600314 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:57.600369 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0721 22:14:57.624421 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:57.627235 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.627328 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.666281 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.771188 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.772628 59970 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0721 22:14:57.772927 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.773188 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:57.773367 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:57.773313 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:57.773439 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.773422 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0721 22:14:57.773651 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:57.773807 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:57.773875 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.773802 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.sharedsubnet.example.com" }] } I0721 22:14:57.773930 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.774001 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0721 22:14:57.774012 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:57.774102 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0721 22:14:57.774155 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:57.774210 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:57.774259 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0721 22:14:57.774326 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0721 22:14:57.774287 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0721 22:14:57.776307 59970 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0721 22:14:57.776625 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0721 22:14:57.776571 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0721 22:14:57.776695 59970 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0721 22:14:57.776761 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0721 22:14:57.776899 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0721 22:14:57.777004 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0721 22:14:57.776945 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0721 22:14:57.777192 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:57.777303 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:57.777353 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.777394 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:57.777499 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:57.777384 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0721 22:14:57.777545 59970 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I0721 22:14:57.777625 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:57.779375 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:57.779518 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.783532 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0721 22:14:57.786331 59970 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0721 22:14:57.786545 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.786731 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.787001 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.787261 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" }] }] } I0721 22:14:57.787423 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0721 22:14:57.787671 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.787846 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.788061 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.788194 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.788405 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.788559 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.788724 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.788785 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0721 22:14:57.788975 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0721 22:14:57.789132 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.789332 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.789533 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.789771 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0721 22:14:57.791541 59970 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0721 22:14:57.791830 59970 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0721 22:14:57.791897 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.791935 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.792000 59970 dns.go:238] Pre-creating DNS records I0721 22:14:57.792027 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.798211 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:57.801447 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.801544 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.842048 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.949523 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.951045 59970 executor.go:111] Tasks: 0 done / 79 total; 43 can run I0721 22:14:57.951362 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:57.951528 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.951652 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:57.951795 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.952067 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:57.952200 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:57.952246 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:57.952324 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.953614 59970 executor.go:111] Tasks: 43 done / 79 total; 16 can run I0721 22:14:57.953773 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0721 22:14:57.953917 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0721 22:14:57.954055 59970 iamrole.go:163] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0721 22:14:57.953950 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0721 22:14:57.954166 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:57.954227 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.954337 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0721 22:14:57.954492 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:57.958475 59970 executor.go:111] Tasks: 59 done / 79 total; 18 can run I0721 22:14:57.958841 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.959057 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.959190 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.959364 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:57.961307 59970 executor.go:111] Tasks: 77 done / 79 total; 2 can run I0721 22:14:57.961542 59970 executor.go:111] Tasks: 79 done / 79 total; 0 can run I0721 22:14:57.964345 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:57.964406 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:57.964475 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:57.964517 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:57.964552 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:57.964583 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:57.964617 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:57.964652 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:57.964689 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:57.964728 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:57.964760 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:57.964789 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:57.964819 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:57.964853 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:57.964886 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:57.964922 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:57.964951 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:57.964982 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:57.965020 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:57.965055 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:57.966462 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:57.966489 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:57.966528 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966581 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0721 22:14:57.966636 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966694 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966734 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966775 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966818 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966852 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0721 22:14:57.966890 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966932 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0721 22:14:57.966965 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0721 22:14:57.967007 59970 api.go:53] PageSize not implemented I0721 22:14:57.967030 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:57.967058 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:57.967077 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:57.967100 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:57.967119 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:57.967168 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:57.967196 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:57.967244 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:57.967276 59970 iamrole.go:91] ListRoles: { } I0721 22:14:57.967320 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:57.967359 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:57.967725 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0721 22:14:57.967826 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:nodes.sharedsubnet.example.com ok keypair:key-1 ok I0721 22:14:57.967850 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0721 22:14:57.967933 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0721 22:14:57.967878 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0721 22:14:57.967974 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } autoscaling-config:lt-2 ok I0721 22:14:57.968019 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok W0721 22:14:57.968004 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:57.968061 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:57.968094 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:57.967950 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok I0721 22:14:57.968147 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0721 22:14:57.968348 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:57.968371 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:57.968417 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:57.968482 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0721 22:14:57.968588 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0721 22:14:57.968645 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0721 22:14:57.968662 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0721 22:14:57.968694 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0721 22:14:57.968721 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0721 22:14:57.968789 59970 iamrole.go:182] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0721 22:14:57.968807 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0721 22:14:57.968838 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0721 22:14:57.969088 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedSubnet I0721 22:14:57.969597 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:57.969644 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:57.969684 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:57.969732 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:57.969770 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:57.969801 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:57.969829 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:57.969861 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:57.969885 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:57.969940 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0721 22:14:58.002042 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:58.005476 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.005656 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.005725 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.044491 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.146187 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.151134 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0721 22:14:58.151438 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0721 22:14:58.151555 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0721 22:14:58.151512 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:58.151679 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:58.151749 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:58.151731 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0721 22:14:58.151832 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.151936 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0721 22:14:58.151936 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:58.152009 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] } I0721 22:14:58.152218 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0721 22:14:58.152129 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:58.152411 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0721 22:14:58.152468 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:58.152477 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0721 22:14:58.152584 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.152589 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0721 22:14:58.152689 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.152904 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0721 22:14:58.153140 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:58.153291 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:58.153348 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-subnet.example.com" }] }] } I0721 22:14:58.153726 59970 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0721 22:14:58.153968 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0721 22:14:58.153978 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0721 22:14:58.154066 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0721 22:14:58.154122 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0721 22:14:58.154206 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0721 22:14:58.154303 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0721 22:14:58.154307 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:58.154340 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0721 22:14:58.154469 59970 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0721 22:14:58.154420 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0721 22:14:58.154577 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0721 22:14:58.154609 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0721 22:14:58.154727 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0721 22:14:58.154820 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0721 22:14:58.155044 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0721 22:14:58.155263 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.155381 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:58.155436 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.155477 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0721 22:14:58.155669 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:58.155755 59970 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Private" },{ Key: "kubernetes.io/role/internal-elb", Value: "1" }] } I0721 22:14:58.155822 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:58.155879 59970 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [{ Key: "kubernetes.io/role/elb", Value: "1" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" },{ Key: "SubnetType", Value: "Utility" }] } I0721 22:14:58.155943 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.156033 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:58.156077 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.156111 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.156219 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:58.156273 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:58.156313 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.156439 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:58.156506 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.156637 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:58.156710 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:58.156757 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:58.156439 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0721 22:14:58.158901 59970 executor.go:111] Tasks: 70 done / 105 total; 31 can run W0721 22:14:58.159209 59970 api.go:53] PageSize not implemented I0721 22:14:58.159117 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W0721 22:14:58.159316 59970 api.go:53] PageSize not implemented I0721 22:14:58.159315 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" }] }] } I0721 22:14:58.159367 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0721 22:14:58.159483 59970 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I0721 22:14:58.159554 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0721 22:14:58.159544 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.159618 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.159726 59970 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0721 22:14:58.159796 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.159866 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0721 22:14:58.159771 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.159961 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0721 22:14:58.160085 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0721 22:14:58.160160 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0721 22:14:58.160033 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:58.160427 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.160708 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I0721 22:14:58.160893 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0721 22:14:58.161032 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" }] }] } I0721 22:14:58.161294 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.161414 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.161528 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.161714 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0721 22:14:58.161845 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.161990 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0721 22:14:58.162130 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0721 22:14:58.162239 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.162356 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.162449 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.162730 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.162911 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.163088 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.163345 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.163486 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0721 22:14:58.163580 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:58.163683 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:58.163804 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.163925 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:58.164058 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" }] }] } I0721 22:14:58.164188 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0721 22:14:58.164985 59970 executor.go:111] Tasks: 101 done / 105 total; 4 can run I0721 22:14:58.165079 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:58.165228 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:58.165487 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:58.165612 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.165667 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.165776 59970 dns.go:238] Pre-creating DNS records I0721 22:14:58.165816 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.171921 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:58.176652 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.176831 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.177001 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.217758 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.323201 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.325624 59970 executor.go:111] Tasks: 0 done / 105 total; 46 can run I0721 22:14:58.325926 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] }] } I0721 22:14:58.326251 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.328182 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.328353 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.326142 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:58.328886 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:58.329066 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:58.329151 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:58.334537 59970 executor.go:111] Tasks: 46 done / 105 total; 24 can run I0721 22:14:58.334738 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0721 22:14:58.334928 59970 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0721 22:14:58.334941 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:58.334965 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0721 22:14:58.335009 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0721 22:14:58.335181 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.335207 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0721 22:14:58.335241 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0721 22:14:58.335352 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:58.335459 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0721 22:14:58.335549 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0721 22:14:58.335699 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.335750 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0721 22:14:58.335911 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:58.335967 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0721 22:14:58.336115 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:58.339991 59970 executor.go:111] Tasks: 70 done / 105 total; 31 can run W0721 22:14:58.340488 59970 api.go:53] PageSize not implemented I0721 22:14:58.340416 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.340617 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.340567 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.340768 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.340856 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } W0721 22:14:58.340980 59970 api.go:53] PageSize not implemented I0721 22:14:58.341003 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.340997 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.341131 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0721 22:14:58.341190 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0721 22:14:58.341216 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.341492 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.341805 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.344308 59970 executor.go:111] Tasks: 101 done / 105 total; 4 can run W0721 22:14:58.344424 59970 api.go:53] PageSize not implemented I0721 22:14:58.344473 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.344587 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0721 22:14:58.344667 59970 api.go:53] PageSize not implemented I0721 22:14:58.344688 59970 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0721 22:14:58.344701 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.344907 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.345003 59970 executor.go:111] Tasks: 105 done / 105 total; 0 can run I0721 22:14:58.348665 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:58.348730 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:58.348765 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:58.348797 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:58.348844 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:58.348875 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:58.348908 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:58.348946 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:58.348985 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:58.349027 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0721 22:14:58.349071 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:58.349106 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:58.349142 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:58.349175 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:58.349211 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:58.349247 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:58.349293 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:58.349330 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:58.349366 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:58.349402 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:58.349439 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:58.349473 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:58.349510 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:58.349547 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:58.351190 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:58.351222 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:58.351263 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351332 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0721 22:14:58.351424 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351487 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351530 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351571 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351612 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351648 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0721 22:14:58.351685 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351732 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0721 22:14:58.351767 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0721 22:14:58.351804 59970 api.go:53] PageSize not implemented I0721 22:14:58.351818 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0721 22:14:58.351870 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:58.351900 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:58.351921 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:58.351945 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:58.351963 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:58.352016 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.352043 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.352091 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:58.352130 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.352177 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:58.352221 59970 routetable.go:53] DescribeRouteTables: { } autoscaling-group:bastion.private-shared-subnet.example.com ok autoscaling-group:nodes.private-shared-subnet.example.com ok autoscaling-config:lt-1 ok I0721 22:14:58.352820 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0721 22:14:58.352877 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.352915 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0721 22:14:58.352899 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok load-balancer:bastion-private-shared-su-5ol32q ok I0721 22:14:58.352990 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0721 22:14:58.352985 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0721 22:14:58.353057 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0721 22:14:58.352969 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0721 22:14:58.353115 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions load-balancer:api-private-shared-subnet-n2f8ak ok I0721 22:14:58.353107 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0721 22:14:58.353143 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:58.353170 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0721 22:14:58.353181 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0721 22:14:58.353067 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-subnet.example.com. ok I0721 22:14:58.353279 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.353339 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0721 22:14:58.353437 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.353459 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:58.353509 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0721 22:14:58.353574 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-3 ok autoscaling-config:lt-2 ok I0721 22:14:58.353645 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:58.353693 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0721 22:14:58.353906 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.353943 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0721 22:14:58.354021 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:58.354090 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0721 22:14:58.354179 59970 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0721 22:14:58.354077 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0721 22:14:58.354213 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.354197 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0721 22:14:58.354233 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0721 22:14:58.354246 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0721 22:14:58.354275 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:58.354287 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0721 22:14:58.354347 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0721 22:14:58.354517 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.354536 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0721 22:14:58.354488 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0721 22:14:58.354597 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0721 22:14:58.354642 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0721 22:14:58.354691 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0721 22:14:58.354757 59970 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0721 22:14:58.354776 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0721 22:14:58.354807 59970 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0721 22:14:58.355091 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.39s) === RUN TestLifecyclePrivateSharedIP I0721 22:14:58.355671 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:58.355721 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:58.355760 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:58.355807 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:58.355844 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:58.355874 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:58.355905 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:58.355934 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:58.355959 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:58.356011 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0721 22:14:58.388213 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:58.391629 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.391737 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.391799 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.431020 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.541361 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.543525 59970 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0721 22:14:58.544066 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0721 22:14:58.544365 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0721 22:14:58.544526 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.544614 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0721 22:14:58.544609 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:58.544767 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0721 22:14:58.544817 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.544990 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0721 22:14:58.545054 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:58.545195 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:58.545090 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0721 22:14:58.545248 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:58.545318 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:58.545343 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:58.545351 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0721 22:14:58.545389 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.545427 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0721 22:14:58.545494 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0721 22:14:58.545489 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.545779 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:58.547050 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:58.547148 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.547797 59970 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0721 22:14:58.548087 59970 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0721 22:14:58.548136 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0721 22:14:58.548289 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0721 22:14:58.548368 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.548605 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0721 22:14:58.548613 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0721 22:14:58.549042 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0721 22:14:58.548907 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0721 22:14:58.548851 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.549366 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0721 22:14:58.549590 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:58.549691 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.549899 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0721 22:14:58.549949 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0721 22:14:58.551162 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.552146 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" } ] }], VpcId: "vpc-12345678" } I0721 22:14:58.552256 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0721 22:14:58.552361 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0721 22:14:58.552391 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0721 22:14:58.552510 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:58.552531 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0721 22:14:58.552606 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.552709 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0721 22:14:58.552826 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.552862 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.552835 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0721 22:14:58.552958 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:58.553014 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.553087 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:58.553140 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:58.553182 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:58.553317 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0721 22:14:58.553389 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:58.553478 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:58.553530 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:58.553578 59970 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0721 22:14:58.553659 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.553751 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:58.553803 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" } ] } I0721 22:14:58.553887 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.553990 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:58.554041 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:58.554077 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0721 22:14:58.554166 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:58.554220 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:58.553834 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0721 22:14:58.554263 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:58.554320 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:58.556400 59970 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0721 22:14:58.556568 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:58.556706 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0721 22:14:58.556795 59970 api.go:53] PageSize not implemented W0721 22:14:58.556829 59970 api.go:53] PageSize not implemented I0721 22:14:58.556861 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0721 22:14:58.556929 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0721 22:14:58.556923 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.557023 59970 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0721 22:14:58.557120 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.557144 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.557254 59970 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0721 22:14:58.557355 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.557285 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.557389 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0721 22:14:58.557460 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0721 22:14:58.557413 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.557549 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0721 22:14:58.557589 59970 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0721 22:14:58.557638 59970 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0721 22:14:58.557633 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.557960 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.558118 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.558249 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.558306 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:58.558382 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.558460 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.558609 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0721 22:14:58.558732 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:58.558900 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0721 22:14:58.559036 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.559167 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0721 22:14:58.559244 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:58.559364 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:58.559433 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I0721 22:14:58.559614 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.559751 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:58.559893 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0721 22:14:58.559937 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:58.559972 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0721 22:14:58.560139 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0721 22:14:58.560233 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:58.560378 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0721 22:14:58.560483 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:58.560610 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0721 22:14:58.560733 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:58.560861 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:58.560912 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.560991 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.561064 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0721 22:14:58.561136 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0721 22:14:58.561273 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0721 22:14:58.561317 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0721 22:14:58.561353 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.561487 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:58.561581 59970 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0721 22:14:58.561666 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.564424 59970 executor.go:111] Tasks: 108 done / 114 total; 5 can run I0721 22:14:58.564636 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:58.564662 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.564785 59970 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:58.564796 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:58.564920 59970 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }] }] } I0721 22:14:58.565014 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0721 22:14:58.565058 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:58.565101 59970 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0721 22:14:58.565165 59970 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0721 22:14:58.565224 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.565347 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0721 22:14:58.565413 59970 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0721 22:14:58.565473 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.565507 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.565560 59970 dns.go:238] Pre-creating DNS records I0721 22:14:58.565586 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.572553 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.7 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.7 ********************************************************************************* I0721 22:14:58.575518 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.575611 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.575675 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.614700 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.723106 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.725112 59970 executor.go:111] Tasks: 0 done / 114 total; 46 can run I0721 22:14:58.725294 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:58.725480 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.725666 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I0721 22:14:58.726775 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.727163 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.728374 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0721 22:14:58.728528 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0721 22:14:58.728571 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0721 22:14:58.732900 59970 executor.go:111] Tasks: 46 done / 114 total; 26 can run I0721 22:14:58.733091 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0721 22:14:58.733118 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I0721 22:14:58.733266 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.733372 59970 iamrole.go:163] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0721 22:14:58.733431 59970 iamrole.go:163] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0721 22:14:58.733425 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0721 22:14:58.733624 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:58.733691 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:58.733787 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0721 22:14:58.733981 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.734049 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0721 22:14:58.734208 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:58.734258 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0721 22:14:58.734403 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0721 22:14:58.734480 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0721 22:14:58.734638 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.734789 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0721 22:14:58.734833 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.734957 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0721 22:14:58.735101 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.738087 59970 executor.go:111] Tasks: 72 done / 114 total; 36 can run I0721 22:14:58.738382 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0721 22:14:58.738448 59970 api.go:53] PageSize not implemented I0721 22:14:58.738526 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.738528 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.738625 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.738667 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0721 22:14:58.738658 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0721 22:14:58.738752 59970 api.go:53] PageSize not implemented I0721 22:14:58.738815 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.738884 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.738923 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0721 22:14:58.738962 59970 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0721 22:14:58.739097 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.739528 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.739643 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:58.739887 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.739988 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0721 22:14:58.740068 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:58.740145 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0721 22:14:58.740268 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0721 22:14:58.741396 59970 executor.go:111] Tasks: 108 done / 114 total; 5 can run W0721 22:14:58.741566 59970 api.go:53] PageSize not implemented I0721 22:14:58.741601 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0721 22:14:58.741599 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.741610 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.741685 59970 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0721 22:14:58.741708 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:58.741711 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0721 22:14:58.741824 59970 api.go:53] PageSize not implemented I0721 22:14:58.741843 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.742028 59970 executor.go:111] Tasks: 113 done / 114 total; 1 can run I0721 22:14:58.742097 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.742198 59970 executor.go:111] Tasks: 114 done / 114 total; 0 can run I0721 22:14:58.746200 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:58.746273 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:58.746324 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:58.746361 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:58.746398 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:58.746431 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:58.746464 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:58.746501 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:58.746539 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:58.746578 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0721 22:14:58.746619 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0721 22:14:58.746656 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:58.746688 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:58.746724 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:58.746762 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:58.746795 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0721 22:14:58.746831 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:58.746867 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:58.746902 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0721 22:14:58.746936 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0721 22:14:58.746987 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0721 22:14:58.747045 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:58.747116 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:58.747180 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0721 22:14:58.747240 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:58.747300 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:58.747365 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:58.747421 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:58.747487 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:58.749225 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:58.749252 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:58.749289 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749352 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0721 22:14:58.749433 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749491 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749531 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749571 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749610 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749652 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0721 22:14:58.749707 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749758 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:58.749785 59970 natgateway.go:117] DescribeNatGateways: { } I0721 22:14:58.749823 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0721 22:14:58.749859 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0721 22:14:58.749898 59970 api.go:53] PageSize not implemented I0721 22:14:58.749916 59970 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0721 22:14:58.749967 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:58.749995 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:58.750013 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:58.750035 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:58.750053 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0721 22:14:58.750121 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.750147 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.750187 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:58.750226 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.750274 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:58.750322 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:58.750356 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0721 22:14:58.750392 59970 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0721 22:14:58.750425 59970 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0721 22:14:58.751066 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0721 22:14:58.751153 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0721 22:14:58.751236 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok W0721 22:14:58.751282 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.751308 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0721 22:14:58.751281 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } autoscaling-config:lt-2 ok I0721 22:14:58.751365 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok autoscaling-group:nodes.private-shared-ip.example.com ok I0721 22:14:58.751376 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:bastion.private-shared-ip.example.com ok I0721 22:14:58.751318 59970 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0721 22:14:58.751411 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } load-balancer:api-private-shared-ip-exa-ohatqj ok I0721 22:14:58.751430 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0721 22:14:58.751463 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0721 22:14:58.751473 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0721 22:14:58.751496 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0721 22:14:58.751541 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok keypair:key-1 ok I0721 22:14:58.751523 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0721 22:14:58.751737 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.751540 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.751759 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok security-group:sg-5 ok I0721 22:14:58.751799 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0721 22:14:58.751853 59970 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0721 22:14:58.751908 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.751962 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0721 22:14:58.752038 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.752053 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-3 ok I0721 22:14:58.752158 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0721 22:14:58.752172 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0721 22:14:58.752231 59970 iamrole.go:182] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0721 22:14:58.752216 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0721 22:14:58.752250 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0721 22:14:58.752292 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0721 22:14:58.752264 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0721 22:14:58.752340 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.752356 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0721 22:14:58.752344 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0721 22:14:58.752408 59970 iamrole.go:182] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0721 22:14:58.752393 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:58.752425 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0721 22:14:58.752438 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.752467 59970 iamrole.go:139] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0721 22:14:58.752500 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0721 22:14:58.752582 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0721 22:14:58.752599 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0721 22:14:58.752638 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0721 22:14:58.752481 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0721 22:14:58.752684 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:58.752708 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:58.752810 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0721 22:14:58.752905 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:58.753215 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0721 22:14:58.753745 59970 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:58.753791 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:58.753832 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:58.753879 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:58.753919 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:58.753948 59970 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:58.753979 59970 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:58.754013 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:58.754039 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:58.754096 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0721 22:14:58.778794 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0721 22:14:58.781634 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.781728 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.822835 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.949925 59970 iamrole.go:91] ListRoles: { } I0721 22:14:58.951514 59970 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0721 22:14:58.951931 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.952188 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:58.952376 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:58.952449 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.953249 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.953413 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:58.953206 59970 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.953574 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.953789 59970 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.952365 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0721 22:14:58.953917 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:58.953986 59970 iamrole.go:55] CreateRole: { AssumeRolePolicyDocument: "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazonaws.com\"},\n \"Action\": \"sts:AssumeRole\"\n }\n ]\n}", RoleName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0721 22:14:58.954002 59970 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 213, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.954781 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0721 22:14:58.954874 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0721 22:14:58.954849 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:58.954904 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0721 22:14:58.954954 59970 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0721 22:14:58.955023 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:58.955098 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:58.955114 59970 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0721 22:14:58.955167 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.955298 59970 iaminstanceprofile.go:99] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0721 22:14:58.955450 59970 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0721 22:14:58.955686 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:58.960423 59970 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0721 22:14:58.960593 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:58.960758 59970 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:14:58.960874 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0721 22:14:58.960815 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0721 22:14:58.961003 59970 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0721 22:14:58.961028 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0721 22:14:58.961143 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0721 22:14:58.961305 59970 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0721 22:14:58.961374 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I0721 22:14:58.961437 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0721 22:14:58.961532 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:58.961615 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:58.961873 59970 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0721 22:14:58.961942 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.962156 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.962138 59970 subnets.go:105] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" } ] }], VpcId: "vpc-1" } I0721 22:14:58.962379 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:58.962455 59970 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.962597 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:58.962663 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:58.962779 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0721 22:14:58.962855 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:58.962956 59970 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0721 22:14:58.962982 59970 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0721 22:14:58.963178 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:58.963279 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:58.963405 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:58.963511 59970 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" } ] } I0721 22:14:58.963142 59970 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeTags\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0721 22:14:58.963669 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:58.965659 59970 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0721 22:14:58.965899 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.966168 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.966399 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.966715 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.966916 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.967158 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.967313 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:58.967400 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.967546 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.967685 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.967801 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.967942 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.968091 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0721 22:14:58.968132 59970 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:58.968183 59970 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0721 22:14:58.968227 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0721 22:14:58.968368 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.968515 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.968619 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.968712 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.968912 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.969066 59970 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0721 22:14:58.969226 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.969375 59970 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0721 22:14:58.969487 59970 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0721 22:14:58.970949 59970 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0721 22:14:58.971241 59970 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0721 22:14:58.971437 59970 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0721 22:14:58.971510 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:58.971546 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.971606 59970 dns.go:238] Pre-creating DNS records I0721 22:14:58.971637 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:58.977284 59970 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.22.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.22.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.13 ********************************************************************************* I0721 22:14:58.980238 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:58.980334 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:59.019619 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:59.147634 59970 iamrole.go:91] ListRoles: { } I0721 22:14:59.149198 59970 executor.go:111] Tasks: 0 done / 98 total; 46 can run I0721 22:14:59.149504 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.149563 59970 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0721 22:14:59.149727 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.149954 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:59.150058 59970 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0721 22:14:59.151350 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.151630 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.151793 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:14:59.151857 59970 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:14:59.151982 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:59.156083 59970 executor.go:111] Tasks: 46 done / 98 total; 23 can run I0721 22:14:59.156269 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.156456 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.156561 59970 iamrole.go:163] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0721 22:14:59.156617 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.156831 59970 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:14:59.156890 59970 iamrole.go:163] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0721 22:14:59.156879 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0721 22:14:59.157051 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:59.157108 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0721 22:14:59.157245 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:59.157315 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:59.157476 59970 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:14:59.161087 59970 executor.go:111] Tasks: 69 done / 98 total; 25 can run I0721 22:14:59.161371 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:59.161547 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:59.161760 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:59.162074 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:59.162277 59970 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0721 22:14:59.162436 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:59.162661 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:59.164183 59970 executor.go:111] Tasks: 94 done / 98 total; 2 can run I0721 22:14:59.164508 59970 executor.go:111] Tasks: 96 done / 98 total; 2 can run I0721 22:14:59.164703 59970 executor.go:111] Tasks: 98 done / 98 total; 0 can run I0721 22:14:59.167719 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0721 22:14:59.167827 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0721 22:14:59.167895 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0721 22:14:59.167960 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0721 22:14:59.168034 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0721 22:14:59.168098 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0721 22:14:59.168160 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0721 22:14:59.168224 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0721 22:14:59.168291 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0721 22:14:59.168353 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0721 22:14:59.168420 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0721 22:14:59.168525 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0721 22:14:59.168616 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0721 22:14:59.168709 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0721 22:14:59.168804 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0721 22:14:59.168883 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:14:59.168968 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0721 22:14:59.169070 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:14:59.169162 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0721 22:14:59.169254 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0721 22:14:59.169345 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0721 22:14:59.169432 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0721 22:14:59.169497 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0721 22:14:59.169558 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:14:59.169619 59970 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0721 22:14:59.171165 59970 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0721 22:14:59.171199 59970 keypairs.go:95] DescribeKeyPairs: { } I0721 22:14:59.171256 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171328 59970 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171415 59970 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171497 59970 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171570 59970 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171650 59970 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171712 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171816 59970 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.171935 59970 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.172066 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0721 22:14:59.172165 59970 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0721 22:14:59.172292 59970 api.go:53] PageSize not implemented I0721 22:14:59.172351 59970 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0721 22:14:59.172420 59970 loadbalancers.go:34] PageSize not implemented I0721 22:14:59.172473 59970 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0721 22:14:59.172534 59970 targetgroups.go:35] PageSize not implemented I0721 22:14:59.172590 59970 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0721 22:14:59.172700 59970 zones.go:93] ListHostedZonesPages { } I0721 22:14:59.172753 59970 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0721 22:14:59.172830 59970 iaminstanceprofile.go:192] ListInstanceProfiles: { } I0721 22:14:59.172881 59970 iamrole.go:91] ListRoles: { } I0721 22:14:59.172945 59970 internetgateways.go:101] DescribeInternetGateways: { } I0721 22:14:59.173010 59970 routetable.go:53] DescribeRouteTables: { } I0721 22:14:59.173065 59970 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0721 22:14:59.173582 59970 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0721 22:14:59.173687 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0721 22:14:59.173684 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0721 22:14:59.173766 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok volume:vol-1 ok I0721 22:14:59.173775 59970 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0721 22:14:59.173825 59970 iaminstanceprofile.go:155] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok autoscaling-config:lt-1 ok I0721 22:14:59.173878 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0721 22:14:59.173926 59970 iaminstanceprofile.go:240] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I0721 22:14:59.173853 59970 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.nthsqsresources.longclustername.example.com. ok I0721 22:14:59.173969 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0721 22:14:59.174145 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:59.174176 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0721 22:14:59.174233 59970 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0721 22:14:59.174321 59970 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0721 22:14:59.174379 59970 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0721 22:14:59.174463 59970 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W0721 22:14:59.174711 59970 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0721 22:14:59.174733 59970 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0721 22:14:59.174836 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0721 22:14:59.174918 59970 iamrole.go:182] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0721 22:14:59.174963 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0721 22:14:59.174937 59970 subnets.go:233] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0721 22:14:59.175429 59970 iamrole.go:139] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0721 22:14:59.175531 59970 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0721 22:14:59.175636 59970 iamrole.go:182] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0721 22:14:59.175652 59970 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0721 22:14:59.175686 59970 iamrole.go:139] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0721 22:14:59.175783 59970 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0721 22:14:59.175871 59970 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0721 22:14:59.175949 59970 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0721 22:14:59.176244 59970 featureflag.go:162] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 53.915s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.047s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0721 22:13:40.969667 54338 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.056s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.038s === RUN TestPodController I0721 22:13:43.043163 55558 pod.go:59] starting pod controller I0721 22:13:43.043590 55558 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.059s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.017s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00019bd60} route53_test.go:93: Got zone list: [0xc00019bd90] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00019be60} route53_test.go:93: Got zone list: [0xc00019be90] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00019bf30} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003aa660 0xc0001a40a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c6090} route53_test.go:93: Got zone list: [0xc0003c60c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a40c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c61c0} route53_test.go:93: Got zone list: [0xc0003c61f0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a40e0} route53_test.go:187: Successfully added resource record set: {0xc00020e100 0xc0001a4100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c6470} route53_test.go:93: Got zone list: [0xc0003c64a0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4188} route53_test.go:199: Successfully added resource record set: {0xc00020e280 0xc0001a41a8} route53_test.go:118: Got 1 recordsets: [0xc0003c6640] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c6770} route53_test.go:93: Got zone list: [0xc0003c67a0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4240} route53_test.go:221: Successfully added resource record set: {0xc00020e400 0xc0001a4260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc00020e400 0xc0001a4260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c6b00} route53_test.go:93: Got zone list: [0xc0003c6b30] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4318} route53_test.go:246: Successfully removed resource set {0xc00020e600 0xc0001a4338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c6db0} route53_test.go:93: Got zone list: [0xc0003c6de0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a43c0} route53_test.go:264: Successfully removed resource set {0xc00020e780 0xc0001a43e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c7090} route53_test.go:93: Got zone list: [0xc0003c70c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4470} commontests.go:192: Successfully added resource record set: {0xc00020e900 0xc0001a4490} commontests.go:45: Correctly replaced resource record {0xc00020e900 0xc0001a4490} -> {0xc00020ea00 0xc0001a44e0} commontests.go:142: Got 1 recordsets: [0xc0003c73d0] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c75a0} route53_test.go:93: Got zone list: [0xc0003c75d0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a4590} commontests.go:192: Successfully added resource record set: {0xc00020ec80 0xc0001a45b0} commontests.go:70: Correctly replaced resource record {0xc00020ec80 0xc0001a45b0} -> {0xc00020ed80 0xc0001a4600} commontests.go:142: Got 1 recordsets: [0xc0003c78e0] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c7b00} route53_test.go:93: Got zone list: [0xc0003c7b30] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a46c0} commontests.go:192: Successfully added resource record set: {0xc00020f000 0xc0001a46e0} commontests.go:142: Got 2 recordsets: [0xc0003c7da0 0xc0003c7dc0] commontests.go:142: Got 2 recordsets: [0xc0003c7e30 0xc0003c7e50] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003c7ff0} route53_test.go:93: Got zone list: [0xc0003d6020] route53_test.go:98: Got at least 1 zone in list:&{0xc0003aa480 0xc0001a47e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.035s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.048s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee200} clouddns_test.go:71: Got zone list: [0xc0005ecb88] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ecb58} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee300} clouddns_test.go:71: Got zone list: [0xc0005ecc30] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ecc00} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee400} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0002fed80 0xc0005eccd8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee4c0} clouddns_test.go:71: Got zone list: [0xc0005ecd80] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ecd50} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee5e0} clouddns_test.go:71: Got zone list: [0xc0005ecea0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ece70} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee720} clouddns_test.go:71: Got zone list: [0xc0005ecff0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ecfc0} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee880} clouddns_test.go:71: Got zone list: [0xc0005ed1e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ed1b8} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ee9e0} clouddns_test.go:71: Got zone list: [0xc0005ed368] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ed338} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005eeb20} clouddns_test.go:71: Got zone list: [0xc0005ed4b8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ed488} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005eec80} clouddns_test.go:71: Got zone list: [0xc0005ed668] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ed638} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005eede0} clouddns_test.go:71: Got zone list: [0xc0005ed920] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005ed8f0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005eef60} clouddns_test.go:71: Got zone list: [0xc0005edc38] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005edc08} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000594040 0xc0005ef140} clouddns_test.go:71: Got zone list: [0xc00000e030] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0002feb00 0xc0005edfc8} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.096s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.035s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.193s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0721 22:13:59.860258 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.860322 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.860352 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.860389 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.860422 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.860435 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.860452 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.860488 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.860501 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.866426 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0721 22:13:59.869480 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.869504 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.869522 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.869543 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.869561 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.869582 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.869595 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.869609 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.869621 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.870388 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.00s) === RUN TestContainerdBuilder_Simple I0721 22:13:59.872947 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.872967 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.872990 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.873012 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.873028 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.873040 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.873062 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.873075 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.873084 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.873845 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.877133 59775 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0721 22:13:59.877155 59775 assetstore.go:111] bin/containerd I0721 22:13:59.877162 59775 assetstore.go:111] bin/containerd-shim I0721 22:13:59.877168 59775 assetstore.go:111] bin/containerd-shim-runc-v1 I0721 22:13:59.877173 59775 assetstore.go:111] bin/containerd-shim-runc-v2 I0721 22:13:59.877178 59775 assetstore.go:111] bin/containerd-stress I0721 22:13:59.877184 59775 assetstore.go:111] bin/ctr I0721 22:13:59.877216 59775 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0721 22:13:59.877225 59775 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0721 22:13:59.880095 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.880126 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.880147 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.880173 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.880195 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.880208 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.880222 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.880239 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.880252 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.881141 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.882242 59775 containerd.go:57] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I0721 22:13:59.883124 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.883151 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.883168 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.883189 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.883246 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.883261 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.883275 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.883295 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.883306 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.884075 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.885190 59775 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0721 22:13:59.885820 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.885840 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.885861 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.885883 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.885916 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.885931 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.885945 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.885959 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.885971 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.886754 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.887918 59775 assetstore.go:108] Matching assets for "^docker/": I0721 22:13:59.887932 59775 assetstore.go:111] docker/containerd I0721 22:13:59.887939 59775 assetstore.go:111] docker/containerd-shim I0721 22:13:59.887946 59775 assetstore.go:111] docker/ctr I0721 22:13:59.887951 59775 assetstore.go:111] docker/docker I0721 22:13:59.887956 59775 assetstore.go:111] docker/docker-init I0721 22:13:59.887962 59775 assetstore.go:111] docker/docker-proxy I0721 22:13:59.887967 59775 assetstore.go:111] docker/dockerd I0721 22:13:59.887972 59775 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.00s) === RUN TestDockerBuilder_LogFlags I0721 22:13:59.890303 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.890333 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.890351 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.890374 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.890409 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.890424 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.890438 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.890452 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.890463 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.891265 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.892343 59775 assetstore.go:108] Matching assets for "^docker/": I0721 22:13:59.892356 59775 assetstore.go:111] docker/containerd I0721 22:13:59.892362 59775 assetstore.go:111] docker/containerd-shim I0721 22:13:59.892367 59775 assetstore.go:111] docker/ctr I0721 22:13:59.892373 59775 assetstore.go:111] docker/docker I0721 22:13:59.892378 59775 assetstore.go:111] docker/docker-init I0721 22:13:59.892383 59775 assetstore.go:111] docker/docker-proxy I0721 22:13:59.892388 59775 assetstore.go:111] docker/dockerd I0721 22:13:59.892393 59775 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.00s) === RUN TestDockerBuilder_SkipInstall I0721 22:13:59.894483 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.894525 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.894542 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.894569 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.894601 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.894615 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.894629 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.894650 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.894661 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.895397 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.896460 59775 docker.go:60] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0721 22:13:59.896669 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.896687 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.896704 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.896723 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.896753 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.896766 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.896787 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.896800 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.896810 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.897499 59775 zones.go:93] ListHostedZonesPages { } I0721 22:13:59.898561 59775 assetstore.go:108] Matching assets for "^docker/": I0721 22:13:59.898575 59775 assetstore.go:111] docker/docker I0721 22:13:59.898582 59775 assetstore.go:111] docker/docker-containerd I0721 22:13:59.898587 59775 assetstore.go:111] docker/docker-containerd-ctr I0721 22:13:59.898593 59775 assetstore.go:111] docker/docker-containerd-shim I0721 22:13:59.898598 59775 assetstore.go:111] docker/docker-init I0721 22:13:59.898603 59775 assetstore.go:111] docker/docker-proxy I0721 22:13:59.898608 59775 assetstore.go:111] docker/docker-runc I0721 22:13:59.898613 59775 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0721 22:13:59.903550 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.903589 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.903613 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.903638 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.903659 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.903672 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.903687 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.903703 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.903716 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.905498 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0721 22:13:59.907285 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.907312 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.907332 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.907354 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.907371 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.907385 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.907399 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.907414 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.907425 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.908687 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0721 22:13:59.910540 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.910570 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.910589 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.910610 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.910626 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.910640 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.910653 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.910667 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.910679 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.911950 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.00s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I0721 22:13:59.916591 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.916633 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.916652 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.916674 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.916692 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.916707 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.916722 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.916736 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.916748 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.918050 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0721 22:13:59.925383 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.925419 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.925439 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.925463 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.925483 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.925496 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.925510 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.925526 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.925538 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.928166 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0721 22:13:59.933209 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.933259 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.933281 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.933304 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.933324 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.933337 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.933351 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.933367 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.933379 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.934623 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0721 22:13:59.939514 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.939558 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.939576 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.939597 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.939614 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.939626 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.939640 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.939655 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.939667 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.940993 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0721 22:13:59.949173 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.949214 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:59.949239 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:59.949264 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:59.949285 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:59.949300 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:59.949314 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:59.949332 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:59.949345 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:59.950685 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.54s) === RUN TestKubeAPIServerBuilderARM64 I0721 22:14:01.492293 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.492346 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.492375 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.492408 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.492437 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.492454 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.492473 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.492493 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.492509 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.494153 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.07s) === RUN TestKubeControllerManagerBuilder I0721 22:14:01.562277 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.562329 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.562359 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.562392 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.562417 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.562434 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.562451 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.562472 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.562487 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.564133 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0721 22:14:01.574887 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.574958 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.575006 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.575051 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.575088 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.575118 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.575148 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.575181 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.575207 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.577920 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0721 22:14:01.646934 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.646988 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.647023 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.647054 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.647079 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.647096 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.647114 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.647135 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.647149 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.648855 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.08s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0721 22:14:01.722738 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.722789 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.722818 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.722846 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.722873 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.722890 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.722909 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.722929 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.722944 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.724613 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0721 22:14:01.728460 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.728508 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.728533 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.728564 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.728587 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.728603 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.728621 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.728639 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.728654 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.730273 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0721 22:14:01.796423 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.796480 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.796515 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.796549 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.796574 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.796598 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.796619 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.796639 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.796654 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.798315 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0721 22:14:01.863740 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.863792 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.863820 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.863851 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.863876 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.863893 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.863910 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.863931 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.863947 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.865611 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0721 22:14:01.875094 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.875239 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.875305 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.875357 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.875401 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.875436 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.875467 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.875503 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.875529 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.878391 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0721 22:14:01.944478 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:01.944531 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:01.944560 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:01.944590 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:01.944613 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:01.944629 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:01.944645 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:01.944664 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:01.944680 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:01.946299 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0721 22:14:02.014715 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.014795 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.014843 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.014891 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.014933 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.014964 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.014993 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.015028 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.015054 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.017757 59775 zones.go:93] ListHostedZonesPages { } I0721 22:14:02.020787 59775 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0721 22:14:02.025663 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.025717 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.025751 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.025784 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.025810 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.025827 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.025845 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.025867 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.025883 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.027387 59775 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0721 22:14:02.031139 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.031184 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.031210 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.031237 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.031259 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.031276 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.031294 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.031312 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.031327 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.032561 59775 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0721 22:14:02.035919 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.035959 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.035981 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.036007 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.036030 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.036045 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.036061 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.036078 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.036092 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.037605 59775 zones.go:93] ListHostedZonesPages { } I0721 22:14:02.039143 59775 assetstore.go:108] Matching assets for "protokube$": I0721 22:14:02.039164 59775 assetstore.go:111] /opt/kops/bin/protokube I0721 22:14:02.039174 59775 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0721 22:14:02.039194 59775 assetstore.go:108] Matching assets for "channels$": I0721 22:14:02.039202 59775 assetstore.go:111] /opt/kops/bin/channels I0721 22:14:02.039210 59775 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I0721 22:14:02.040743 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.040785 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.040808 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.040834 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.040857 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.040875 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.040892 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.040910 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.040924 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.042455 59775 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0721 22:14:02.044753 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.044791 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.044815 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.044841 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.044863 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.044881 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.044896 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.044912 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.044927 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.046426 59775 zones.go:93] ListHostedZonesPages { } I0721 22:14:02.047922 59775 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0721 22:14:02.048419 59775 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:02.048460 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:02.048486 59775 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:02.048513 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:02.048536 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:02.048553 59775 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:02.048570 59775 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:02.048587 59775 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:02.048602 59775 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:02.050500 59775 zones.go:93] ListHostedZonesPages { } I0721 22:14:02.052704 59775 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.245s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.051s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.120s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.078s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.103s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0721 22:13:43.198073 55647 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0721 22:13:43.198220 55647 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.054s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0721 22:13:52.048846 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049267 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049337 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049388 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049442 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0721 22:13:52.049701 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049776 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049824 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049882 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049941 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.049988 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050039 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050099 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050149 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050201 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050250 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050312 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050359 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050408 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050454 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050521 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050578 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050624 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0721 22:13:52.050671 58077 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0721 22:13:52.050889 58077 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0721 22:13:52.050970 58077 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvdia --- PASS: Test_Validate_Nvdia (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.098s ? k8s.io/kops/pkg/apis/nodeup [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0721 22:13:44.752100 56016 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0721 22:13:44.752148 56016 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0721 22:13:44.752156 56016 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0721 22:13:44.752172 56016 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.099s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.048s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.174s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.068s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.01s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.178s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.101s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.014s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.055s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.117s === RUN TestFlagToFalse I0721 22:13:42.977536 55535 featureflag_test.go:33] Created flag Unittest1 I0721 22:13:42.977608 55535 featureflag.go:162] FeatureFlag "UnitTest1"=false I0721 22:13:42.977624 55535 featureflag.go:162] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0721 22:13:42.977708 55535 featureflag.go:162] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.034s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.106s === RUN TestRenderInstanceGroupZones W0721 22:13:45.826848 56403 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.056s === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:100: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0721 22:14:13.270151 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.270234 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.271407 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.271432 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.276676 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.276699 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.276710 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.276717 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.280318 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.281772 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.281805 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.281826 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.283037 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.283064 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.288251 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.288294 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:13.288983 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:13.289016 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.289036 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.290225 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.290260 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.295554 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.295588 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.295605 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.295617 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.296566 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.297207 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.297237 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.297257 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.298461 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.298497 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.303783 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.303833 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.304764 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.304798 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.304823 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.306038 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.306072 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.311308 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.311359 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.312237 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.312270 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.312294 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.313495 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.313533 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.318812 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.318843 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.318864 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.318879 60026 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0721 22:14:13.319923 60026 instancegroups.go:398] Draining the node: "node-2a.local". I0721 22:14:13.320760 60026 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0721 22:14:13.320807 60026 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0721 22:14:13.320833 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.322036 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.322072 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.327332 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.327382 60026 instancegroups.go:398] Draining the node: "node-2b.local". I0721 22:14:13.328337 60026 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0721 22:14:13.328372 60026 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0721 22:14:13.328395 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.329601 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.329636 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.334876 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.334927 60026 instancegroups.go:398] Draining the node: "node-2c.local". I0721 22:14:13.335869 60026 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0721 22:14:13.335902 60026 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0721 22:14:13.335925 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.337132 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.337166 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.342417 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.342453 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0721 22:14:13.343105 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.343137 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.344333 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.344392 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.344437 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.344478 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.344503 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.345705 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.345756 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.345772 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.345797 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.347013 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.347051 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.347097 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.347112 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.347137 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.348340 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.348400 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.348439 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.348468 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.349711 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.349787 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.349809 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.349846 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.351070 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.351100 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.351159 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.351177 60026 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0721 22:14:13.351204 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.352464 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.352519 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.352535 60026 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0721 22:14:13.352563 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.353771 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. W0721 22:14:13.353823 60026 instancegroups.go:395] Not draining cluster nodes as 'cloudonly' flag is set. I0721 22:14:13.353839 60026 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0721 22:14:13.353865 60026 instancegroups.go:435] waiting for 1ms after terminating instance W0721 22:14:13.355063 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0721 22:14:13.355098 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0721 22:14:13.355504 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.355535 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.356741 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.356783 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.356809 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.356892 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.356919 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.356936 60026 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.356949 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.357676 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.358574 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.358608 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.358633 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.359880 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.359925 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.359948 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.359991 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:13.360992 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:13.361030 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.361056 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.362291 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.362333 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.362355 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.362378 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.362395 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.362408 60026 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.362422 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.363491 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.364270 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.364306 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.364330 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.365569 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.365610 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.365632 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.365673 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.366520 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.366551 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.366574 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.367801 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.367854 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.367876 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.367918 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.368891 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.368927 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.368950 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.370177 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.370217 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.370239 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.370254 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.370272 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.370285 60026 instancegroups.go:477] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.370300 60026 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0721 22:14:13.371496 60026 instancegroups.go:398] Draining the node: "node-2a.local". I0721 22:14:13.372318 60026 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0721 22:14:13.372349 60026 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0721 22:14:13.372372 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.373567 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.373608 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.373629 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.373671 60026 instancegroups.go:398] Draining the node: "node-2b.local". I0721 22:14:13.374578 60026 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0721 22:14:13.374611 60026 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0721 22:14:13.374635 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.375859 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.375900 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.375921 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.375963 60026 instancegroups.go:398] Draining the node: "node-2c.local". I0721 22:14:13.376854 60026 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0721 22:14:13.376885 60026 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0721 22:14:13.376926 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.378125 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.378166 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. W0721 22:14:13.378213 60026 instancegroups.go:477] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0721 22:14:13.378234 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0721 22:14:13.378513 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0721 22:14:13.378802 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.378832 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.380074 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.380110 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.385364 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.385430 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.385452 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.385465 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.386176 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.386992 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.387026 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.387049 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.388258 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.388293 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.393550 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.393602 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:13.394485 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:13.394523 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.394547 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.395781 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.395816 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.401076 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.401136 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.401159 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.401173 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.402214 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.402963 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.402997 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.403020 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.404247 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.404281 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.409553 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.409604 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.410467 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.410501 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.410524 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.411738 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.411774 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.417064 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.417117 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.417962 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.417996 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.418022 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.419259 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.419294 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.424771 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.424804 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.424825 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.424839 60026 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0721 22:14:13.425831 60026 instancegroups.go:398] Draining the node: "node-2a.local". I0721 22:14:13.426577 60026 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0721 22:14:13.426610 60026 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0721 22:14:13.426633 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.427867 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.427903 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.433175 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.433227 60026 instancegroups.go:398] Draining the node: "node-2b.local". I0721 22:14:13.434078 60026 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0721 22:14:13.434112 60026 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0721 22:14:13.434135 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.435383 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.435418 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.440678 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.440731 60026 instancegroups.go:398] Draining the node: "node-2c.local". I0721 22:14:13.441591 60026 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0721 22:14:13.441626 60026 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0721 22:14:13.441649 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.442885 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.442920 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.448173 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.448209 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0721 22:14:13.448342 60026 rollingupdate.go:96] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0721 22:14:13.448824 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.448855 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.450084 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.450125 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.450146 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0721 22:14:13.450540 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.450570 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.451790 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.451828 60026 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0721 22:14:13.451851 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0721 22:14:13.452107 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.452136 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.452152 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0721 22:14:13.452348 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.452376 60026 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0721 22:14:13.452392 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0721 22:14:13.452639 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.452668 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.453893 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.453935 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.459208 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.459272 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.459297 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.459311 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.460053 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.460869 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.460903 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.460926 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.462171 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.462221 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.462242 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0721 22:14:13.462521 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.462550 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.463757 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.463799 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.469065 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.469129 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.469155 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.469175 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.469979 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.470769 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.470802 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.470827 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.472033 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.472076 60026 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0721 22:14:13.472098 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0721 22:14:13.472372 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.472398 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.472412 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.473401 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.474130 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.474160 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.474184 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.475385 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.475423 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.480690 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.480741 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.481678 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.481711 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.481734 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.482937 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.482979 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.488232 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.488282 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.489077 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.489107 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.489130 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.490351 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.490387 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.495635 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.495669 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0721 22:14:13.495947 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.495976 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.495994 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0721 22:14:13.496217 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.496243 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.496258 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0721 22:14:13.497383 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.497410 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.497425 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0721 22:14:13.497585 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.497615 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.497628 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.498609 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.499428 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.499460 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.499483 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.500689 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.500736 60026 instancegroups.go:521] Cluster did not pass validation within deadline: testing failure. E0721 22:14:13.500757 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0721 22:14:13.500976 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.501006 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.501019 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.502075 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.502817 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.502850 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.502873 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.504098 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.504140 60026 instancegroups.go:511] Cluster did not validate within deadline: testing validation error. E0721 22:14:13.504162 60026 instancegroups.go:473] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0721 22:14:13.504465 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.504496 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.505701 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.505751 60026 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0721 22:14:13.506956 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.512265 60026 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0721 22:14:13.513452 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.518728 60026 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0721 22:14:13.519910 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.525166 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.525249 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.525276 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.525293 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.526062 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.526872 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.526905 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.526931 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.528136 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.528181 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.533507 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.533565 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:13.534378 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:13.534413 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.534437 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.535682 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.535785 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.541048 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.541089 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.541143 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.541161 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.542152 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.542903 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.542939 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.542962 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.544193 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.544242 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.549799 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.549854 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.550700 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.550734 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.550758 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.551991 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.552039 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.557327 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.557381 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.558222 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.558256 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.558279 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.559518 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.559566 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.564825 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.564859 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.564888 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.564903 60026 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0721 22:14:13.565927 60026 instancegroups.go:398] Draining the node: "node-2a.local". I0721 22:14:13.566684 60026 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0721 22:14:13.566717 60026 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0721 22:14:13.566740 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.567960 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.568009 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.573317 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.573371 60026 instancegroups.go:398] Draining the node: "node-2b.local". I0721 22:14:13.574211 60026 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0721 22:14:13.574246 60026 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0721 22:14:13.574269 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.575495 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.575548 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.580853 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.580908 60026 instancegroups.go:398] Draining the node: "node-2c.local". I0721 22:14:13.581773 60026 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0721 22:14:13.581807 60026 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0721 22:14:13.581831 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.583077 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.583125 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.588436 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.588472 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0721 22:14:13.588766 60026 instancegroups.go:591] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0721 22:14:13.588796 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.590025 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.590068 60026 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0721 22:14:13.591202 60026 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0721 22:14:13.592415 60026 instancegroups.go:524] Cluster did not pass validation, will retry in "1ms": testing failure. I0721 22:14:13.593590 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.598875 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.598942 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.598978 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.598993 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.599758 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.600566 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.600597 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.600621 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.601847 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.601884 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.607163 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.607217 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:13.608069 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:13.608101 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.608125 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.609328 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.609365 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.614619 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.614659 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.614683 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.614698 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.615613 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.616396 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.616428 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.616451 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.617674 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.617711 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.622872 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.622926 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.623769 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.623800 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.623823 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.625053 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.625091 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.630342 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.630398 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.631259 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.631294 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.631318 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.632561 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.632597 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.637853 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.637885 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.637907 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.637922 60026 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0721 22:14:13.638891 60026 instancegroups.go:398] Draining the node: "node-2a.local". I0721 22:14:13.640506 60026 instancegroups.go:417] deleting node "node-2a.local" from kubernetes I0721 22:14:13.640551 60026 instancegroups.go:589] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0721 22:14:13.640573 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.641725 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.641777 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.646940 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.647007 60026 instancegroups.go:398] Draining the node: "node-2b.local". I0721 22:14:13.647936 60026 instancegroups.go:417] deleting node "node-2b.local" from kubernetes I0721 22:14:13.647968 60026 instancegroups.go:589] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0721 22:14:13.647992 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.649213 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.649250 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.654398 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.654466 60026 instancegroups.go:398] Draining the node: "node-2c.local". I0721 22:14:13.655311 60026 instancegroups.go:417] deleting node "node-2c.local" from kubernetes I0721 22:14:13.655342 60026 instancegroups.go:589] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0721 22:14:13.655365 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.656579 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.656616 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.661863 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.661903 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0721 22:14:13.662634 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.662666 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.662679 60026 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0721 22:14:13.663465 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.664264 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.664299 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.664321 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.665555 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.665590 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.670855 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.670907 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.671770 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.671808 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.671832 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.673067 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.673101 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.678349 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.678382 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0721 22:14:13.678649 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.678676 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.678689 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.679411 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:13.680156 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:13.680190 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:13.680215 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.681411 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.681448 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.686713 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.686764 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:13.687584 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:13.687615 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:13.687638 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.688855 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.688891 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.694144 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.694181 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0721 22:14:13.694523 60026 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled W0721 22:14:13.694565 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0721 22:14:13.694584 60026 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled W0721 22:14:13.694599 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0721 22:14:13.694617 60026 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled W0721 22:14:13.694627 60026 instancegroups.go:466] Not validating cluster as cloudonly flag is set. I0721 22:14:13.694644 60026 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0721 22:14:13.694657 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0721 22:14:13.694885 60026 instancegroups.go:568] Detaching instance "bastion-1a", in group "bastion-1". I0721 22:14:13.694938 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:13.696135 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.696170 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.701399 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.701436 60026 instancegroups.go:183] Rolling updates for InstanceGroup bastion-1 are disabled I0721 22:14:13.701497 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.701526 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.701546 60026 instancegroups.go:309] Tainting 2 nodes in "master-1" instancegroup. I0721 22:14:13.702272 60026 instancegroups.go:183] Rolling updates for InstanceGroup master-1 are disabled I0721 22:14:13.702306 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.702325 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.702338 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:13.703313 60026 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0721 22:14:13.703376 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:13.704582 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.704616 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.709892 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.709928 60026 instancegroups.go:183] Rolling updates for InstanceGroup node-1 are disabled I0721 22:14:13.709944 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.709963 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.709979 60026 instancegroups.go:309] Tainting 3 nodes in "node-2" instancegroup. I0721 22:14:13.710947 60026 instancegroups.go:566] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0721 22:14:13.711004 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:13.712257 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.712291 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:13.717556 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.717590 60026 instancegroups.go:183] Rolling updates for InstanceGroup node-2 are disabled I0721 22:14:13.717608 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0721 22:14:13.717883 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.717918 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.717934 60026 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0721 22:14:13.720077 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.720828 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.720858 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.720890 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.722118 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.722166 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.722254 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:13.722255 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.723744 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:13.723780 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:13.724158 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.724195 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.724228 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.725466 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.725535 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.826114 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.826168 60026 instancegroups.go:398] Draining the node: "node-1e.local". I0721 22:14:13.826201 60026 instancegroups.go:398] Draining the node: "node-1d.local". I0721 22:14:13.827526 60026 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0721 22:14:13.827560 60026 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0721 22:14:13.827864 60026 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0721 22:14:13.827893 60026 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0721 22:14:13.827923 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.829121 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.829170 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.829214 60026 instancegroups.go:398] Draining the node: "node-1f.local". I0721 22:14:13.830038 60026 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0721 22:14:13.830071 60026 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0721 22:14:13.830102 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.831311 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.831374 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.931961 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.932010 60026 instancegroups.go:398] Draining the node: "node-1g.local". I0721 22:14:13.932907 60026 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0721 22:14:13.932941 60026 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0721 22:14:13.932972 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.934202 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.934247 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.934266 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0721 22:14:13.934512 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.934550 60026 instancegroups.go:501] Cluster validated. I0721 22:14:13.934564 60026 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0721 22:14:13.936364 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:13.936435 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:13.937631 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:13.937667 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:13.937955 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:13.937989 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:13.938018 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:13.939257 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:13.939322 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.039913 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.039967 60026 instancegroups.go:398] Draining the node: "node-1d.local". I0721 22:14:14.040002 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:14.041401 60026 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0721 22:14:14.041437 60026 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0721 22:14:14.041774 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:14.041808 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:14.041839 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.043056 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.043099 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.043140 60026 instancegroups.go:398] Draining the node: "node-1e.local". I0721 22:14:14.043952 60026 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0721 22:14:14.043985 60026 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0721 22:14:14.044016 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.045218 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.045286 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.145914 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.145965 60026 instancegroups.go:398] Draining the node: "node-1f.local". I0721 22:14:14.146858 60026 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0721 22:14:14.146893 60026 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0721 22:14:14.146924 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.148121 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.148165 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.148183 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0721 22:14:14.148393 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.148425 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.148440 60026 instancegroups.go:309] Tainting 7 nodes in "master-1" instancegroup. I0721 22:14:14.150513 60026 instancegroups.go:398] Draining the node: "master-1a.local". I0721 22:14:14.151266 60026 instancegroups.go:417] deleting node "master-1a.local" from kubernetes I0721 22:14:14.151302 60026 instancegroups.go:589] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0721 22:14:14.151333 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.152536 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.152582 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.152626 60026 instancegroups.go:398] Draining the node: "master-1c.local". I0721 22:14:14.152668 60026 instancegroups.go:398] Draining the node: "master-1b.local". I0721 22:14:14.153998 60026 instancegroups.go:417] deleting node "master-1c.local" from kubernetes I0721 22:14:14.154034 60026 instancegroups.go:589] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0721 22:14:14.154378 60026 instancegroups.go:417] deleting node "master-1b.local" from kubernetes I0721 22:14:14.154411 60026 instancegroups.go:589] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0721 22:14:14.154464 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.155674 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.155747 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.256410 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.256462 60026 instancegroups.go:398] Draining the node: "master-1e.local". I0721 22:14:14.256491 60026 instancegroups.go:398] Draining the node: "master-1d.local". I0721 22:14:14.257832 60026 instancegroups.go:417] deleting node "master-1e.local" from kubernetes I0721 22:14:14.258113 60026 instancegroups.go:417] deleting node "master-1d.local" from kubernetes I0721 22:14:14.258149 60026 instancegroups.go:589] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0721 22:14:14.258183 60026 instancegroups.go:589] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0721 22:14:14.258246 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.259469 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.259513 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.259553 60026 instancegroups.go:398] Draining the node: "master-1f.local". I0721 22:14:14.260387 60026 instancegroups.go:417] deleting node "master-1f.local" from kubernetes I0721 22:14:14.260422 60026 instancegroups.go:589] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0721 22:14:14.260453 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.261678 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.261747 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.362391 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.362441 60026 instancegroups.go:398] Draining the node: "master-1g.local". I0721 22:14:14.363261 60026 instancegroups.go:417] deleting node "master-1g.local" from kubernetes I0721 22:14:14.363294 60026 instancegroups.go:589] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0721 22:14:14.363326 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.364546 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.364590 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.364608 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0721 22:14:14.364844 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.364878 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.364892 60026 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0721 22:14:14.366686 60026 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0721 22:14:14.366751 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.367934 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.367978 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.367991 60026 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0721 22:14:14.368043 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.369249 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.369292 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.369339 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:14.369408 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:14.370725 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:14.370763 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:14.371079 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:14.371139 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:14.371176 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.372381 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.372442 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.472967 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.473021 60026 instancegroups.go:398] Draining the node: "node-1d.local". I0721 22:14:14.473106 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:14.474352 60026 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0721 22:14:14.474387 60026 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0721 22:14:14.474719 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:14.474752 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:14.474783 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.475961 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.476015 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.476068 60026 instancegroups.go:398] Draining the node: "node-1e.local". I0721 22:14:14.476960 60026 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0721 22:14:14.476997 60026 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0721 22:14:14.477047 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.478261 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.478324 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.578869 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.578945 60026 instancegroups.go:398] Draining the node: "node-1f.local". I0721 22:14:14.579894 60026 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0721 22:14:14.579927 60026 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0721 22:14:14.579972 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.581206 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.581280 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.581306 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0721 22:14:14.582492 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.582561 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.582607 60026 instancegroups.go:309] Tainting 6 nodes in "node-1" instancegroup. I0721 22:14:14.585721 60026 instancegroups.go:566] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0721 22:14:14.585781 60026 instancegroups.go:566] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0721 22:14:14.585826 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.587041 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.587075 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.587108 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:14.587146 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:14.588036 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:14.588061 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:14.588251 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:14.588271 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:14.588291 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.589476 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.589558 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.690182 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.690266 60026 instancegroups.go:398] Draining the node: "node-1d.local". I0721 22:14:14.690369 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:14.691914 60026 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0721 22:14:14.691953 60026 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0721 22:14:14.692314 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:14.692358 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:14.692404 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.693635 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.693684 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.693728 60026 instancegroups.go:398] Draining the node: "node-1e.local". I0721 22:14:14.694616 60026 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0721 22:14:14.694650 60026 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0721 22:14:14.694688 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.695892 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.695969 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.796854 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.796930 60026 instancegroups.go:398] Draining the node: "node-1f.local". I0721 22:14:14.797911 60026 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0721 22:14:14.797948 60026 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0721 22:14:14.797991 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.799242 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.799308 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.799696 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0721 22:14:14.800012 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.800039 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.800056 60026 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0721 22:14:14.800841 60026 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0721 22:14:14.800876 60026 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0721 22:14:14.800915 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.802120 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.802160 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:14.807381 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.807441 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:14.807481 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:14.808839 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:14.808880 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:14.808907 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.809229 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:14.809270 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:14.809295 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.810083 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.810117 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:14.815378 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.815414 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I0721 22:14:14.815708 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.815738 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.815753 60026 instancegroups.go:309] Tainting 2 nodes in "node-1" instancegroup. I0721 22:14:14.816452 60026 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0721 22:14:14.816501 60026 instancegroups.go:566] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0721 22:14:14.816542 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.817731 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.817766 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:14.823040 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.823097 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:14.823115 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:14.824413 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:14.824464 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:14.824491 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.824781 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:14.824820 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:14.824845 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.825725 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.825759 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:14.831198 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.831250 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0721 22:14:14.831581 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.831634 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.831656 60026 instancegroups.go:309] Tainting 4 nodes in "node-1" instancegroup. I0721 22:14:14.833295 60026 instancegroups.go:566] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0721 22:14:14.833360 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.834551 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.834597 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.834612 60026 instancegroups.go:566] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0721 22:14:14.834669 60026 instancegroups.go:170] waiting for 1ms after detaching instance I0721 22:14:14.835855 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.835901 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.835954 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:14.836030 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:14.836082 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:14.837742 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:14.837796 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:14.837834 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.838129 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:14.838167 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:14.838203 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.838504 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:14.838543 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:14.838578 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.839061 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.939809 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.939888 60026 instancegroups.go:398] Draining the node: "node-1d.local". I0721 22:14:14.940918 60026 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0721 22:14:14.940956 60026 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0721 22:14:14.940997 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.942228 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.942292 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.942311 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0721 22:14:14.942593 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.942626 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.942642 60026 instancegroups.go:309] Tainting 7 nodes in "node-1" instancegroup. I0721 22:14:14.944812 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:14.945581 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:14.945620 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:14.945655 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.946857 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.946903 60026 instancegroups.go:501] Cluster validated. I0721 22:14:14.946946 60026 instancegroups.go:398] Draining the node: "node-1e.local". I0721 22:14:14.947023 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:14.948305 60026 instancegroups.go:417] deleting node "node-1e.local" from kubernetes I0721 22:14:14.948343 60026 instancegroups.go:589] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0721 22:14:14.948649 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:14.948686 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:14.948722 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:14.949949 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:14.950011 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.050868 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.050955 60026 instancegroups.go:398] Draining the node: "node-1g.local". I0721 22:14:15.051014 60026 instancegroups.go:398] Draining the node: "node-1f.local". I0721 22:14:15.052390 60026 instancegroups.go:417] deleting node "node-1g.local" from kubernetes I0721 22:14:15.052441 60026 instancegroups.go:589] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0721 22:14:15.052756 60026 instancegroups.go:417] deleting node "node-1f.local" from kubernetes I0721 22:14:15.052793 60026 instancegroups.go:589] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0721 22:14:15.052828 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.054044 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.054098 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.054142 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:15.055093 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:15.055125 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:15.055158 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.056379 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.056451 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.157279 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.157378 60026 instancegroups.go:398] Draining the node: "node-1d.local". I0721 22:14:15.158961 60026 instancegroups.go:417] deleting node "node-1d.local" from kubernetes I0721 22:14:15.159008 60026 instancegroups.go:589] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0721 22:14:15.159066 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.160295 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.160354 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.160374 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0721 22:14:15.160636 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.160664 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.160684 60026 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0721 22:14:15.160706 60026 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0721 22:14:15.160724 60026 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0721 22:14:15.160740 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0721 22:14:15.160914 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.160944 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.160957 60026 instancegroups.go:309] Tainting 3 nodes in "node-1" instancegroup. I0721 22:14:15.162108 60026 instancegroups.go:131] deleting warm pool instance "node-1-wp-a" I0721 22:14:15.162147 60026 instancegroups.go:131] deleting warm pool instance "node-1-wp-b" I0721 22:14:15.162166 60026 instancegroups.go:131] deleting warm pool instance "node-1-wp-c" I0721 22:14:15.162206 60026 instancegroups.go:398] Draining the node: "node-1a.local". I0721 22:14:15.163070 60026 instancegroups.go:417] deleting node "node-1a.local" from kubernetes I0721 22:14:15.163105 60026 instancegroups.go:589] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0721 22:14:15.163129 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.164309 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.164356 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:15.169705 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.169800 60026 instancegroups.go:398] Draining the node: "node-1b.local". I0721 22:14:15.171203 60026 instancegroups.go:417] deleting node "node-1b.local" from kubernetes I0721 22:14:15.171252 60026 instancegroups.go:589] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0721 22:14:15.171289 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.172531 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.172567 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:15.177831 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.177889 60026 instancegroups.go:398] Draining the node: "node-1c.local". I0721 22:14:15.178768 60026 instancegroups.go:417] deleting node "node-1c.local" from kubernetes I0721 22:14:15.178803 60026 instancegroups.go:589] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0721 22:14:15.178826 60026 instancegroups.go:435] waiting for 1ms after terminating instance I0721 22:14:15.180060 60026 instancegroups.go:468] Validating the cluster. I0721 22:14:15.180095 60026 instancegroups.go:504] Cluster validated; revalidating in 5ms to make sure it does not flap. I0721 22:14:15.185346 60026 instancegroups.go:501] Cluster validated. I0721 22:14:15.185381 60026 rollingupdate.go:210] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.994s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.038s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) PASS ok k8s.io/kops/pkg/k8scodecs 0.049s === RUN TestParse E0721 22:13:44.628745 55969 versions.go:96] unable to parse Kubernetes version "" E0721 22:13:44.628818 55969 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.048s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.063s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.20s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.05s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.04s) PASS ok k8s.io/kops/pkg/kubeconfig 0.380s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.125s ? k8s.io/kops/pkg/model/alimodel [no test files] === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.058s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.083s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.141s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/pollinterval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0721 22:13:55.950405 59183 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0721 22:13:55.950564 59183 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 W0721 22:13:55.952009 59183 model.go:347] overriding log level in manifest (embedded etcd manifest), new level is 3 W0721 22:13:55.952175 59183 model.go:511] overloading ENV var in manifest (embedded etcd manifest) with ETCD_QUOTA_BACKEND_BYTES=10737418240 --- PASS: Test_RunEtcdManagerBuilder (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/pollinterval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.106s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.089s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.073s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.067s === RUN TestServerGroupBuilder --- PASS: TestServerGroupBuilder (0.24s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.301s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.050s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.048s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.061s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.45s) --- PASS: TestIssueCert/ca (0.37s) --- PASS: TestIssueCert/client (0.28s) --- PASS: TestIssueCert/clientOneYear (0.80s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTrip --- PASS: TestPrivateKeyRoundTrip (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.465s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.028s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/ali [no test files] === RUN TestAddUntaggedRouteTables I0721 22:13:57.904991 59596 routetable.go:53] DescribeRouteTables: { } W0721 22:13:57.905105 59596 tags.go:67] cluster tag not found on route-table:rtb-1234 I0721 22:13:57.905121 59596 aws.go:224] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListRouteTables I0721 22:13:57.905203 59596 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0721 22:13:57.905296 59596 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0721 22:13:57.905405 59596 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0721 22:13:57.905511 59596 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0721 22:13:57.905567 59596 volumes.go:128] DescribeVolumes: { } W0721 22:13:57.905610 59596 aws.go:2236] (new) cluster tag not found on volume:vol-2 aws_test.go:192: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0721 22:13:57.905995 59596 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:57.906035 59596 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:13:57.906100 59596 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:13:57.906174 59596 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:13:57.906226 59596 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:13:57.906253 59596 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:13:57.906290 59596 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:13:57.906325 59596 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:13:57.906346 59596 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:13:57.906373 59596 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0721 22:13:57.906425 59596 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0721 22:13:57.906472 59596 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0721 22:13:57.906548 59596 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0721 22:13:57.906585 59596 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0721 22:13:57.906629 59596 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0721 22:13:57.906677 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0721 22:13:57.906708 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0721 22:13:57.906736 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0721 22:13:57.906762 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0721 22:13:57.906791 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0721 22:13:57.906829 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0721 22:13:57.906861 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0721 22:13:57.906885 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0721 22:13:57.906926 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0721 22:13:57.906954 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0721 22:13:57.906984 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0721 22:13:57.907008 59596 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.058s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.091s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.185s ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.028s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.005s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.028s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.004s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.03s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.01s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.01s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.119s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.056s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/ali [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.020s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W0721 22:13:48.543695 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543778 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543789 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543798 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543813 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543822 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543832 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543843 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543854 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543863 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543873 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543883 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543893 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543904 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.543914 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.543925 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.546429 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.546485 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.546570 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.546588 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.546677 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.546700 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.546793 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.546814 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.546883 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.546900 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.546996 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547018 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547107 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547123 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547212 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547245 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547318 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547334 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547413 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547429 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547508 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547525 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547612 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547633 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547701 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547748 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547832 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547851 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.547937 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.547992 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548092 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548113 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548172 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548188 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548264 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548280 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548428 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548446 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548530 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548546 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548644 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548661 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548754 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548771 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.548850 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.548867 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549256 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549272 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549334 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549362 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549442 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549458 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549542 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549559 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549632 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549648 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549753 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549770 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549843 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549873 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.549936 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.549965 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550068 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550097 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550179 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550196 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550257 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550272 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550351 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550367 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550447 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550463 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550568 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550584 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550655 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550689 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550754 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550784 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550861 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550877 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.550941 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.550956 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551014 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551029 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551099 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551140 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551211 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551226 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551312 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551327 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551391 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551406 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551473 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551489 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551559 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551583 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551642 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551657 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551728 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551744 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551814 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551829 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.551891 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.551906 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552021 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552039 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552114 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552130 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552202 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552217 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552299 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552315 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552374 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552389 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552457 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552473 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552548 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552583 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552650 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552666 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552729 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552745 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552815 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552830 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552897 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.552913 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.552993 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553014 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553072 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553087 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553150 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553165 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553243 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553260 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553327 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553344 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553411 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553426 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553491 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553506 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553576 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553592 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553662 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553677 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553746 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553762 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553828 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553843 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553911 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.553926 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.553987 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554009 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554074 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554093 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554158 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554179 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554250 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554269 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554346 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554367 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554425 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554440 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554525 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554545 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554605 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554620 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554682 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554697 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554762 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554777 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554853 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554869 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.554940 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.554956 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555025 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555040 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555099 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555119 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555193 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555209 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555277 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555293 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555358 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555374 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555434 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555450 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555520 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555536 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555603 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555619 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555687 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555708 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555769 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555785 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555852 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555867 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.555926 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.555941 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0721 22:13:48.556553 57501 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0721 22:13:48.556568 57501 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0721 22:13:48.556578 57501 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0721 22:13:48.556587 57501 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0721 22:13:48.556596 57501 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0721 22:13:48.556603 57501 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0721 22:13:48.556611 57501 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0721 22:13:48.556645 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556661 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556671 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.556680 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556690 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.556700 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556711 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.556721 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556732 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.556746 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556758 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556768 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556780 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556790 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.556799 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.556809 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.565397 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.565435 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.565513 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.565561 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.565632 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.565649 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.565719 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.565736 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.565810 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.565827 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.565924 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.565940 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566006 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566029 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566098 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566114 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566177 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566193 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566270 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566287 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566345 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566374 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566448 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566464 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566542 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566558 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566622 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566639 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566721 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566742 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566804 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566820 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566890 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566906 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.566976 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.566993 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567059 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567075 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567150 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567167 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567231 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567247 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567308 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567323 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567394 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567416 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567516 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567532 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567599 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567614 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567688 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567705 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567762 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567792 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567867 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567892 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.567957 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.567973 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568032 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568048 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568154 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568171 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568239 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568262 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568327 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568343 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568406 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568422 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568490 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568505 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568578 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568594 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568657 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568680 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568779 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568795 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568868 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568895 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.568963 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.568978 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569083 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569101 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569194 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569211 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569291 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569307 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569378 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569393 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569459 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569474 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569534 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569550 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569608 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569629 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569695 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569710 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569778 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569793 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569860 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569875 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.569938 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.569954 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570029 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570044 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570102 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570117 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570181 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570196 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570249 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570269 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570418 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570435 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570496 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570510 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570577 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570592 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570645 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570659 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570729 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570743 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570801 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570815 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570894 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570908 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.570965 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.570980 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571053 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571069 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571159 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571177 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571243 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571262 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571320 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571341 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571404 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571420 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571482 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571497 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571551 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571566 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571626 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571650 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571755 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571771 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571827 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571840 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571899 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571913 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.571971 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.571985 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572050 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572064 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572129 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572144 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572208 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572223 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572287 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572309 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572369 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572384 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572448 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572463 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572528 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572544 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572614 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572629 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572695 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572717 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572783 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572798 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572858 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572872 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.572945 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.572964 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573033 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573049 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573117 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573132 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573202 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573217 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573280 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573295 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573364 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573379 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573444 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573458 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573516 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573531 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573624 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573640 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573712 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573734 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573804 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573819 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573880 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573896 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0721 22:13:48.573963 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0721 22:13:48.573979 57501 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.040s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/hostmount [no test files] ? k8s.io/kops/protokube/pkg/protokube [no test files] === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.081s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.02s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.03s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.02s) --- PASS: TestChannelImages/stable-channel (0.01s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.127s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 --- PASS: TestConversionMinimal (0.01s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) PASS ok k8s.io/kops/tests/integration/conversion 0.079s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0721 22:13:53.754610 58407 files.go:57] Writing file "/tmp/fitest1603159632/SubDir/test1.tmp" I0721 22:13:53.754803 58407 files.go:113] Changing file mode for "/tmp/fitest1603159632/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.071s === RUN TestBootstrapChannelBuilder_BuildTasks I0721 22:14:00.893591 59817 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:00.893672 59817 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:00.893700 59817 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:00.893741 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:00.893774 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:00.893787 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:00.893807 59817 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:00.893830 59817 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:00.893843 59817 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:00.899475 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:00.940913 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.006531 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.078235 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.135424 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.211711 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.271277 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.339825 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.459070 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:01.560361 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:02.684732 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:03.715770 59817 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (2.88s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0721 22:14:03.775879 59817 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:03.775938 59817 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:03.775969 59817 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:03.776000 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:03.776028 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:03.776044 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:03.776062 59817 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:03.776084 59817 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:03.776100 59817 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:03.776158 59817 featureflag.go:165] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0721 22:14:03.777355 59817 zones.go:93] ListHostedZonesPages { } I0721 22:14:03.835896 59817 featureflag.go:165] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.06s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0721 22:14:03.836263 59817 internetgateways.go:66] CreateInternetGateway: { } I0721 22:14:03.836335 59817 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0721 22:14:03.836365 59817 subnets.go:180] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0721 22:14:03.836397 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0721 22:14:03.836425 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0721 22:14:03.836442 59817 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0721 22:14:03.836461 59817 iamrole.go:55] CreateRole: { RoleName: "kops-custom-node-role" } I0721 22:14:03.836481 59817 iaminstanceprofile.go:55] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0721 22:14:03.836497 59817 iaminstanceprofile.go:127] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0721 22:14:03.837709 59817 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.09s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.18s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.12s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0721 22:14:04.258557 59817 new_cluster.go:935] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_Kubenet --- PASS: TestPopulateCluster_Kubenet (0.00s) === RUN TestPopulateCluster_CNI --- PASS: TestPopulateCluster_CNI (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:28: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:43: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0721 22:14:04.311039 59817 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0721 22:14:04.311078 59817 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0721 22:14:04.311097 59817 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0721 22:14:04.311145 59817 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0721 22:14:04.311166 59817 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet a I0721 22:14:04.311196 59817 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet b I0721 22:14:04.311228 59817 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet b I0721 22:14:04.311259 59817 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0721 22:14:04.311279 59817 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0721 22:14:04.311310 59817 subnets.go:182] Assigned CIDR 10.32.0.0/11 to subnet a I0721 22:14:04.311328 59817 subnets.go:182] Assigned CIDR 10.64.0.0/11 to subnet b I0721 22:14:04.311342 59817 subnets.go:182] Assigned CIDR 10.96.0.0/11 to subnet b I0721 22:14:04.311358 59817 subnets.go:182] Assigned CIDR 10.128.0.0/11 to subnet c I0721 22:14:04.311373 59817 subnets.go:182] Assigned CIDR 10.160.0.0/11 to subnet c I0721 22:14:04.311391 59817 subnets.go:196] Assigned CIDR 10.0.0.0/14 to subnet a I0721 22:14:04.311409 59817 subnets.go:196] Assigned CIDR 10.4.0.0/14 to subnet b I0721 22:14:04.311428 59817 subnets.go:196] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-amd64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/images/protokube-linux-arm64 (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 3.490s ? k8s.io/kops/upup/pkg/fi/cloudup/alitasks [no test files] === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/aliup 0.116s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0721 22:13:59.763191 59757 target.go:112] Terraform output is in /tmp/kops-render-1024719731 I0721 22:13:59.764468 59757 target.go:112] Terraform output is in /tmp/kops-render-1024719731 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestAutoscalingGroupCloudformationRender I0721 22:13:59.765271 59757 target.go:168] Cloudformation output is in /tmp/kops-render-2164075276 --- PASS: TestAutoscalingGroupCloudformationRender (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0721 22:13:59.765774 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0721 22:13:59.765844 59757 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0721 22:13:59.765886 59757 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0721 22:13:59.766032 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.766095 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.766146 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.766161 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.766263 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.766289 59757 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0721 22:13:59.766320 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.766369 59757 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0721 22:13:59.766432 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0721 22:13:59.766527 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.766566 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.766600 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.766612 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.766636 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.766664 59757 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0721 22:13:59.766705 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0721 22:13:59.766905 59757 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0721 22:13:59.766964 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.767050 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0721 22:13:59.767085 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.767104 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.767124 59757 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0721 22:13:59.767218 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.767350 59757 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0721 22:13:59.767392 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.767410 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.767429 59757 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0721 22:13:59.767471 59757 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.767579 59757 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0721 22:13:59.767621 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.767642 59757 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0721 22:13:59.767671 59757 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0721 22:13:59.767793 59757 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0721 22:13:59.767827 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.767882 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.767896 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.767914 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.767931 59757 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0721 22:13:59.767959 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.767997 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.768021 59757 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0721 22:13:59.768047 59757 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.768081 59757 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0721 22:13:59.768099 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0721 22:13:59.768125 59757 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0721 22:13:59.768229 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0721 22:13:59.768253 59757 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0721 22:13:59.768268 59757 internetgateways.go:66] CreateInternetGateway: { } I0721 22:13:59.768286 59757 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0721 22:13:59.768383 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.768414 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.768442 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.768453 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.768495 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.768526 59757 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0721 22:13:59.768556 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.768605 59757 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0721 22:13:59.768658 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0721 22:13:59.768751 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.768790 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.768825 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.768839 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.768862 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.768889 59757 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0721 22:13:59.768930 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateCloudformationRender I0721 22:13:59.769470 59757 target.go:168] Cloudformation output is in /tmp/kops-render-3110608979 I0721 22:13:59.769648 59757 target.go:168] Cloudformation output is in /tmp/kops-render-3110608979 --- PASS: TestLaunchTemplateCloudformationRender (0.00s) === RUN TestLaunchTemplateTerraformRender I0721 22:13:59.771337 59757 target.go:112] Terraform output is in /tmp/kops-render-1926808184 I0721 22:13:59.772730 59757 target.go:112] Terraform output is in /tmp/kops-render-1926808184 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0721 22:13:59.773120 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.773161 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.773235 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0721 22:13:59.773267 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.773285 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.773303 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.773370 59757 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0721 22:13:59.773516 59757 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0721 22:13:59.773558 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0721 22:13:59.773584 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0721 22:13:59.773681 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.773714 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.773747 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.773759 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.773775 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.773792 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.773818 59757 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0721 22:13:59.773868 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0721 22:13:59.774135 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.774166 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.774231 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0721 22:13:59.774265 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.774293 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.774316 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.774346 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.774415 59757 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0721 22:13:59.774472 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.774489 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.774508 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0721 22:13:59.774613 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.774644 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.774675 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.774686 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.774702 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.774722 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.774749 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.774785 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.774802 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0721 22:13:59.774994 59757 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0721 22:13:59.775024 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.775083 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0721 22:13:59.775113 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.775133 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.775151 59757 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0721 22:13:59.775182 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.775263 59757 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0721 22:13:59.775315 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.775334 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.775359 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.775542 59757 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:13:59.775607 59757 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0721 22:13:59.775842 59757 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0721 22:13:59.775881 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.775922 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.775935 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.775961 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.775983 59757 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0721 22:13:59.776021 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.776078 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.776094 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.776128 59757 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0721 22:13:59.777031 59757 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0721 22:13:59.777093 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.777591 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0721 22:13:59.777636 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.777655 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.777686 59757 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0721 22:13:59.777729 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.777797 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.777895 59757 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0721 22:13:59.777880 59757 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0721 22:13:59.777950 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.777976 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.777997 59757 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0721 22:13:59.778135 59757 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0721 22:13:59.778171 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.778206 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.778219 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.778240 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.778266 59757 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0721 22:13:59.778300 59757 subnets.go:123] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0721 22:13:59.778363 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.778450 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.778471 59757 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0721 22:13:59.778611 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0721 22:13:59.778679 59757 subnets.go:105] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0721 22:13:59.778809 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.778842 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.778871 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.778886 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.778936 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.778961 59757 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0721 22:13:59.778991 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.779016 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.779085 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0721 22:13:59.779103 59757 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0721 22:13:59.779125 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0721 22:13:59.779228 59757 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0721 22:13:59.779258 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.779287 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.779298 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.779321 59757 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0721 22:13:59.779349 59757 subnets.go:123] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0721 22:13:59.779387 59757 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0721 22:13:59.779558 59757 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0721 22:13:59.779595 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.779658 59757 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0721 22:13:59.779686 59757 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0721 22:13:59.779702 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.779718 59757 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0721 22:13:59.779806 59757 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0721 22:13:59.779836 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0721 22:13:59.779866 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.779877 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.779895 59757 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0721 22:13:59.779911 59757 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0721 22:13:59.780044 59757 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0721 22:13:59.780144 59757 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0721 22:13:59.780175 59757 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0721 22:13:59.780206 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0721 22:13:59.780217 59757 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0721 22:13:59.780239 59757 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.058s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.043s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.055s === RUN TestDiskRenderAzure I0721 22:13:55.777869 59159 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0721 22:13:55.778284 59159 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0721 22:13:55.778529 59159 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0721 22:13:55.779038 59159 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0721 22:13:55.779272 59159 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0721 22:13:55.779531 59159 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0721 22:13:55.779744 59159 resourcegroup.go:103] Creating a new Resource Group with name: rg I0721 22:13:55.779769 59159 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0721 22:13:55.780000 59159 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0721 22:13:55.780749 59159 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0721 22:13:55.781006 59159 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0721 22:13:55.781315 59159 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0721 22:13:55.781672 59159 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0721 22:13:55.781999 59159 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.064s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.063s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gcetasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.02s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.060s === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.125s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.088s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.047s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.024s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0721 22:14:01.732103 59929 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0721 22:14:01.732185 59929 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0721 22:14:01.732196 59929 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0721 22:14:01.732207 59929 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0721 22:14:01.732218 59929 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0721 22:14:01.732233 59929 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0721 22:14:01.732243 59929 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0721 22:14:01.732651 59929 service.go:93] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.054s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.016s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.026s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.033s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.035s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.023s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W0721 22:13:57.282002 59501 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.019s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.011s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.018s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.004s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.025s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar I0721 22:14:02.725168 59953 s3context.go:89] Found S3_ENDPOINT="foo.s3.amazonaws.com", using as non-AWS S3 backend W0721 22:14:02.725222 59953 s3context.go:237] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.060s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... kubernetes so:libc.musl-x86_64.so.1 >>> kops*: Package size: 156.6 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.23.0-r3.apk >>> kops: Build complete at Thu, 21 Jul 2022 22:15:03 +0000 elapsed time 0h 3m 24s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20220721.221141) (2/3) Purging kubernetes (1.24.3-r1) (3/3) Purging go (1.18.4-r2) Executing busybox-1.35.0-r19.trigger OK: 333 MiB in 101 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...